Automatic training method and device of video AI algorithm model based on bytecode enhancement

Abstract

The application relates to an automatic training method and device for a video AI algorithm model based on bytecode enhancement, wherein the method comprises the following steps: based on a preset bytecode enhancement proxy mechanism and a bytecode enhancement library, a data backflow proxy is constructed, and context data dynamically intercepted is transmitted to a dispatcher based on the data backflow proxy; in the dispatcher, corresponding image data is acquired according to the context data and a mapping relationship; intelligent labeling is performed on the image data to obtain training data; and according to a preset training strategy, the training data and the mapping relationship, an original video AI algorithm model is automatically trained to obtain a target video AI algorithm model. Through the application, the problems that in the related art, a data backflow link is long, efficiency is low, application invasiveness is strong, development cost is high, and maintenance and upgrading are not facilitated are solved, non-invasive data backflow is realized, development cost is reduced, maintenance and upgrading are facilitated, the data backflow link is reduced, and efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video analysis, in particular to an automatic training method and device for a video AI algorithm model based on bytecode enhancement. BACKGROUND

[0002] With the wide application of artificial intelligence technology in the field of video analysis, such as security monitoring, behavior analysis, industrial quality inspection, etc., the accuracy and recall rate of AI algorithm models have become key indicators to measure their performance. However, in actual application, after the AI algorithm model is deployed to the production environment, it will often cause false positives or false negatives due to factors such as scene changes, lighting conditions, and target object differences, so the algorithm model needs to be optimized and iterated. When false positives or samples that need to be optimized are generated in the traditional optimization and iteration process (such as user manual confirmation of alarms), manual collection and labeling of these pictures or video data is required, and then manual import to the training platform is required. The entire process involves multiple roles and systems, and the process is cumbersome, time-consuming and labor-intensive. Moreover, if automatic data backflow is to be achieved, the function of data reporting usually needs to be developed by hard coding in the video analysis application system. This not only increases the development cost, but also causes strong coupling between the algorithm training system and the business application system, which is not conducive to independent maintenance and upgrade of each system.

[0003] Currently, there is no effective solution to the problem of long data backflow link in related technologies, resulting in low efficiency, strong application invasiveness, high development cost, and poor maintenance and upgrade. SUMMARY

[0004] An automatic training method and device for a video AI algorithm model based on bytecode enhancement are provided in the present embodiment to solve the problem of long data backflow link in related technologies, resulting in low efficiency, strong application invasiveness, high development cost, and poor maintenance and upgrade.

[0005] In a first aspect, an automatic training method for a video AI algorithm model based on bytecode enhancement is provided in the present embodiment, comprising:

[0006] Based on a preset bytecode enhancement proxy mechanism and bytecode enhancement library, a data backflow proxy is constructed, and context data dynamically intercepted is transmitted to a scheduler based on the data backflow proxy; the bytecode enhancement proxy mechanism has a plug-in architecture, and the plug-in architecture is used for isolated loading with a target application; the scheduler is configured with a mapping relationship between algorithms, resources and tasks;

[0007] In the scheduler, corresponding image data is obtained according to the context data and the mapping relationship; and the image data is intelligently labeled to obtain training data;

[0008] According to the preset training strategy, the training data, and the mapping relationship, the original video AI algorithm model is automatically trained to obtain a target video AI algorithm model.

[0009] In some embodiments, the bytecode enhancement agent mechanism further includes:

[0010] The data backflow agent is loaded through a start parameter or an environment variable of the virtual machine; a dynamic bytecode enhancement strategy; and an asynchronous data reporting strategy.

[0011] In some embodiments, the asynchronous data reporting strategy includes:

[0012] The context data is transmitted to the dispatcher in a non-blocking I / O or multi-thread concurrent processing manner.

[0013] In some embodiments, the image data is intelligently labeled to obtain training data, including:

[0014] An intelligent labeling service is called to intelligently label the image data.

[0015] The image data labeled as error samples is recorded in a non-target example library.

[0016] The image data labeled as correct samples is recorded in a target example library to obtain training data.

[0017] In some embodiments, according to the preset training strategy, the training data, and the mapping relationship, the original video AI algorithm model is automatically trained to obtain a target video AI algorithm model, including:

[0018] When at least one trigger condition in the preset training strategy is met, a new round of automatic training is triggered; the trigger conditions in the training strategy are that the backflow context data reaches a preset quantity threshold, the time of the last round of automatic training exceeds a preset time threshold, and a target trigger signal is received.

[0019] According to the training data and the mapping relationship, the original video AI algorithm model is automatically trained to obtain a target video AI algorithm model.

[0020] In some embodiments, according to the training data and the mapping relationship, the original video AI algorithm model is automatically trained to obtain a target video AI algorithm model, including:

[0021] Based on a preset data loading mode and the mapping relationship, the training data is obtained; the data loading mode includes a single-process mode and a multi-process mode.

[0022] Load the original video AI algorithm model and set the corresponding training parameters;

[0023] According to the training data and the training parameters, the original video AI algorithm model is automatically trained, and the preset key indicators in the automatic training process are continuously monitored to automatically train the target video AI algorithm model.

[0024] In some embodiments, the method further comprises:

[0025] According to the unique identifier and the mapping relationship in the context data, the target video AI algorithm model is evaluated to obtain an evaluation result;

[0026] When the evaluation result is an evaluation pass, the corresponding target video AI algorithm model is converted according to the deployment environment, and the converted target video AI algorithm model is deployed to a target server.

[0027] In a second aspect, the present embodiment provides an automatic training device for a video AI algorithm model based on bytecode enhancement, comprising a construction module, an acquisition and labeling module, and a training module.

[0028] The construction module is configured to construct a data backflow agent based on a preset bytecode enhancement proxy mechanism and a bytecode enhancement library, and transmit context data dynamically intercepted to a dispatcher based on the data backflow agent; the bytecode enhancement proxy mechanism has a plug-in architecture, and the plug-in architecture is configured to be loaded in isolation with a target application; the dispatcher is configured with a mapping relationship between algorithms, resources, and tasks;

[0029] The acquisition and labeling module is configured to acquire corresponding image data in the dispatcher according to the context data and the mapping relationship, and intelligently label the image data to obtain training data;

[0030] The training module is configured to automatically train an original video AI algorithm model according to a preset training strategy, the training data, and the mapping relationship, to obtain a target video AI algorithm model.

[0031] In a third aspect, the present embodiment provides a computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the automatic training method for a video AI algorithm model based on bytecode enhancement according to the first aspect.

[0032] In a fourth aspect, the present embodiment provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the automatic training method of the bytecode-enhanced video AI algorithm model as described in the first aspect.

[0033] Compared with the related art, the automatic training method and device of the bytecode-enhanced video AI algorithm model provided in the present embodiment, by means of the preset bytecode-enhanced proxy mechanism and bytecode-enhanced library, constructs a data backflow proxy, and transmits the dynamically intercepted context data to the dispatcher based on the data backflow proxy; the bytecode-enhanced proxy mechanism has a plug-in architecture, which is used for isolated loading with the target application; the dispatcher is configured with the mapping relationship between the algorithm and the resource and the task; in the dispatcher, the corresponding image data is obtained according to the context data and the mapping relationship; and the image data is intelligently annotated to obtain training data; the original video AI algorithm model is automatically trained according to the preset training strategy, the training data and the mapping relationship, to obtain the target video AI algorithm model, solving the problems in the related art that the data backflow link is long, resulting in low efficiency, strong application invasiveness, high development cost, and being not conducive to maintenance and upgrading; the plug-in architecture in the bytecode-enhanced proxy mechanism is used to isolate the target application, realizing non-invasive data backflow, reducing the development cost, facilitating maintenance and upgrading, and automatically dynamically intercepting context data, thereby reducing the data backflow link and improving the efficiency.

[0034] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate the illustrative embodiments of the present application and their description serves to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0036] Figure 1 is a hardware structure block diagram of a terminal device for the automatic training method of the bytecode-enhanced video AI algorithm model provided by an embodiment of the present application;

[0037] Figure 2 is a flowchart of the automatic training method of the bytecode-enhanced video AI algorithm model provided by an embodiment of the present application;

[0038] Figure 3 is a flowchart of step S230;

[0039] Figure 4 is a flowchart of the automatic training method of the bytecode-enhanced video AI algorithm model provided by an embodiment of the present application;

[0040] Figure 5 is a flowchart of an automatic training method of a video AI algorithm model based on bytecode enhancement provided by an embodiment of the present application;

[0041] Figure 6 is a structural block diagram of an automatic training device of a video AI algorithm model based on bytecode enhancement provided by an embodiment of the present application.

[0042] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 210, construction module; 220, acquisition annotation module; 230, training module. DETAILED DESCRIPTION

[0043] In order to more clearly understand the objectives, technical solutions and advantages of the present application, the present application is described and explained below in conjunction with the accompanying drawings and embodiments.

[0044] Unless otherwise defined, technical terms or scientific terms used in the present application shall have the general meanings understood by those skilled in the art with general knowledge. In the present application, "one", "a", "an", "the", "these" and similar words do not represent quantitative limitation, and they can be singular or plural. In the present application, the terms "include", "contain", "have" and any variants thereof are intended to cover non-exclusive inclusion; for example, a process, method and system, product or device containing a series of steps or modules (units) are not limited to the listed steps or modules (units), but can include steps or modules (units) not listed, or can include other steps or modules (units) inherent to the process, method, product or device. In the present application, the terms "connected", "connected", "coupled" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. In the present application, "multiple" means two or more. The association between the associated objects is described by the term "and / or", which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. In general, the character " / " represents an "or" relationship between the objects before and after it. In the present application, the terms "first", "second", "third" and the like are only used to distinguish similar objects, and do not represent a specific order of the objects.

[0045] The method embodiments provided in the present embodiment can be executed in a terminal, a computer or a similar computing device. For example, the method embodiments are run on a terminal, Figure 1 is a hardware structural block diagram of a terminal of an automatic training method of a video AI algorithm model based on bytecode enhancement of the present embodiment. As shown in the figure, Figure 1As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0046] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the automatic training method of the bytecode-enhanced video AI algorithm model in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0048] This embodiment provides an automatic training method for a video AI algorithm model based on bytecode enhancement. Figure 2 This is a flowchart of the automatic training method for the video AI algorithm model based on bytecode enhancement in this embodiment, as shown below. Figure 2 As shown, the process includes the following steps:

[0049] Step S210, based on the preset bytecode enhancement agent mechanism and bytecode enhancement library, a data backflow agent is constructed, and the dynamically intercepted context data is transmitted to the dispatcher based on the data backflow agent; the bytecode enhancement agent mechanism has a plug-in architecture, and the plug-in architecture is used for isolated loading with the target application; the dispatcher is configured with an algorithm respectively mapping the resource and the task;

[0050] Step S220, in the dispatcher, corresponding image data is obtained according to the context data and the mapping relationship; and the image data is intelligently labeled to obtain training data;

[0051] Step S230, according to the preset training strategy, the training data and the mapping relationship, the original video AI algorithm model is automatically trained to obtain a target video AI algorithm model.

[0052] Specifically, bytecode refers to a binary file format generated after Java language compilation, which contains all information of Java program, including class information, method information, variable information, etc. Bytecode is the basis for Java program execution, which is used to realize the loading, execution, debugging and other functions of Java virtual machine (JVM). The bytecode enhancement agent (train-agent) technology refers to a class of technologies for modifying existing bytecode or dynamically generating new bytecode files, which can increase, delete or modify the functions of original methods without modifying the application source code. In the embodiment, the bytecode enhancement agent mechanism has a plug-in architecture, and the plug-in architecture is used for isolated loading with the target application; wherein each plug-in architecture forms a corresponding plug-in, which defines the target class and method to be enhanced (such as AlarmService.confirmAlert or AlarmService.markAsFalsePositive, etc.), and the processing logic to be injected; it is loaded by the self-defined TrainAgentClassLoader to avoid dependency pollution to the target application, so as to achieve the purpose of non-intrusion.

[0053] The bytecode enhancement library is a powerful Java bytecode generation and manipulation library for dynamically creating and modifying Java classes at runtime. Specifically, the bytecode enhancement library can be used with Byte Buddy, ASM, and CGLIB, among others, without limitation. Then, combined with the bytecode enhancement proxy mechanism and the technology of the bytecode enhancement proxy, the data backflow agent can achieve dynamic interception of valuable training data of key methods when the target application starts, obtain context data, and transmit the context data to the dispatcher. The context data includes, but is not limited to, alarm data associated with pictures, video frames, unique identifiers of algorithms, user confirmation behaviors (correct or false alarm), etc. It should be noted that the context data can also be divided into false alarm confirmation and missed alarm addition. When confirming false alarms, the alarm data is inferred by the algorithm, and the algorithm type can be determined from the alarm data, such as fire identification, smoke identification, etc., so that the unique identifier of the algorithm can be determined and integrated into the context data for reporting. When adding missed alarms, the user selects the algorithm and labels the tags. After the user labels, the enhanced proxy intercepts the corresponding method to form the context data of the "correct sample", which includes the unique identifier of the algorithm, and the unique identifier of the algorithm is obtained according to the user-selected algorithm and labeled tags.

[0054] After receiving the context data, the dispatcher first obtains the corresponding image data according to the context data and the mapping relationship, and then intelligently labels the image data to obtain training data. Since the mapping relationship is the relationship between the algorithm and the resource (image data, training task configuration, etc.) and the task, and the context data has a unique identifier of the algorithm, the corresponding image data can be obtained by processing the unique identifier of the algorithm in the context data and the mapping relationship. Intelligent labeling of the image data can automatically and efficiently obtain accurate training data.

[0055] According to the preset training strategy, the training data, and the mapping relationship, the original video AI algorithm model is automatically trained to obtain the target video AI algorithm model. The training strategy includes multiple trigger conditions for determining whether to trigger a new round of automatic training to balance the training resources and the accuracy of the model application. After determining that a new round of automatic training is triggered, the related resources (including training parameters, training tasks, training containers, etc.) of the training can be called through the mapping relationship, and the target data set is combined with the training data to complete the automatic training of the original video AI algorithm model under the related model training method, so as to obtain the target video AI algorithm model that balances the accuracy and recall rate, thereby reducing false alarms. The F1 score (harmonic mean of the two) can be used to comprehensively measure the recall rate and the precision rate. The higher the F1 score, the better the balance between the recall rate and the precision rate of the target video AI algorithm model.

[0056] In the related art, after the AI algorithm model is deployed to the production environment, false positives or false negatives may be caused by factors such as scene changes, lighting conditions, and target object differences, and therefore the algorithm model needs to be optimized and iterated. When false positives or samples that need to be optimized are generated in the traditional optimization and iteration process (such as alarms confirmed manually by users), these pictures or video data need to be collected and labeled manually, and then imported to the training platform manually. The whole process involves multiple roles and systems, and is complicated and time-consuming. Moreover, if automatic data backflow is to be realized, the function of data reporting needs to be developed by hard coding in the video analysis application system. This not only increases the development cost, but also causes strong coupling between the algorithm training system and the business application system, which is not conducive to independent maintenance and upgrade of each system. In the present embodiment, a data backflow agent is constructed based on a preset bytecode enhancement agent mechanism and bytecode enhancement library, and context data dynamically intercepted is transmitted to a dispatcher based on the data backflow agent; the bytecode enhancement agent mechanism has a plug-in architecture, and the plug-in architecture is used for isolated loading with a target application; the dispatcher is configured with a mapping relationship between algorithms and resources and tasks; in the dispatcher, corresponding image data is obtained according to the context data and the mapping relationship; the image data is intelligently labeled to obtain training data; and the original video AI algorithm model is automatically trained according to a preset training strategy, the training data, and the mapping relationship to obtain a target video AI algorithm model, solving the problems in the related art that the data backflow link is long, resulting in low efficiency, and the application is highly invasive, the development cost is high, and maintenance and upgrade are not conducive. The plug-in architecture in the bytecode enhancement agent mechanism is used to isolate the target application, realizing non-invasive data backflow, reducing the development cost, facilitating maintenance and upgrade, and automatically dynamically intercepting context data, thereby reducing the data backflow link and improving the efficiency.

[0057] The above steps are described in detail as follows:

[0058] In the dispatcher, the mapping relationship between algorithms and resources and tasks is configured, as shown in Table 1.

[0059] Table 1

[0060]

[0061] In some embodiments, the bytecode enhancement agent mechanism further includes:

[0062] The data backflow agent is loaded through a start parameter or an environment variable of the virtual machine; a dynamic bytecode enhancement strategy; and an asynchronous data reporting strategy.

[0063] The data backflow agent loaded by the start-up parameter of the virtual machine is: bytecode enhancement agent (train-agent) start-up, an independent configuration file train-agent.conf is read, and configuration override is allowed through the virtual machine start-up parameter or an environmental variable, target addresses of data backflow, authentication information, etc. are determined, so as to complete initialization and configuration loading.

[0064] The dynamic bytecode enhancement strategy is: using AgentBuilder to dynamically match a target class when the virtual machine loads a class according to the definition of the plug-in. When the matching is successful, code is inserted at the entrance or exit of a specified method through a transformer (Transformer). Since the plug-in is isolated from the target application, it can achieve the purpose of non-intrusion.

[0065] The asynchronous data reporting strategy is: the inserted code is responsible for collecting context information, the context information can be structured data, and the structured data is reported to the dispatcher in an asynchronous manner.

[0066] Through the cooperation of the above-mentioned bytecode enhancement agent mechanisms, specifically, the data backflow mechanism based on bytecode enhancement does not need to modify the business application system code, realizes the decoupling of the training system and the business system, is easy to deploy, and has strong universality; has the characteristics of non-intrusion and low coupling.

[0067] In some embodiments, the asynchronous data reporting strategy includes:

[0068] The context data is transmitted to the dispatcher in a non-blocking I / O or multi-thread concurrent processing manner.

[0069] Specifically, the non-blocking I / O processing is: realized through Selector, Channel and Buffer. The Selector is a multiplexer that can monitor the I / O events (such as connection readiness, data readable / writable) of multiple Channels at the same time, avoiding creating independent threads for each connection. The Channel is a bidirectional data channel (such as SocketChannel) that supports non-blocking reading and writing. The Buffer is a data buffer that reduces the frequent switching between user mode and kernel mode. Further, the main thread registers the events of interest (such as OP_WRITE) through the Selector, and when the Channel can write data, the Selector notifies the thread to process, avoiding invalid waiting.

[0070] Multi-thread concurrent processing is: manage background threads through a thread pool (such as ThreadPoolExecutor), avoid the overhead of frequent thread creation / destruction, the main thread encapsulates the upload task as a Runnable or Callable object, submits it to the thread pool, and the worker thread in the thread pool is responsible for executing specific I / O operations (such as sending data through Socket), after the task is completed, the result is processed through Future.get() (blocking acquisition) or the callback method of CompletableFuture (such as thenAccept()).

[0071] Through the embodiment, asynchronous data reporting in multiple modes is realized, so as to further improve the applicability of the method embodiments.

[0072] In other embodiments, other methods can also be used to realize asynchronous data reporting, which is not limited.

[0073] In some of them, the step S220 of intelligently labeling the image data to obtain the training data includes the following steps:

[0074] Step S221, calling an intelligent labeling service to intelligently label the image data;

[0075] Step S222, recording the image data labeled as error samples into a non-target example library;

[0076] Step S223, recording the image data labeled as correct samples into a target example library to obtain the training data.

[0077] Specifically, in the reflux process of the context data, the user's confirmation behavior (correct or false alarm) can be obtained, and based on the confirmation behavior, the context data can be marked as a correct sample or an error sample.

[0078] Among them, the target example library is a correct recognition example library of the target scene; the non-target example library is an incorrect recognition library or a non-target example library.

[0079] The scheduler can call an intelligent labeling service to intelligently label the image data; for the image data labeled as "correct sample", record it into the correct recognition example library of the target scene, so as to form the training data. For "error sample", the intelligent labeling service will try to perform intelligent labeling (for example, pre-labeling by a higher-precision "teacher model" or manual auxiliary tool), if it is a correct sample, record it into the target example library to obtain the training data. If it is still an error sample, record it into the incorrect recognition library or the non-target example library. After all the image data is processed, the corresponding picture state is updated to "automatically labeled".

[0080] Through the embodiment, effective training data is quickly and accurately screened out, thereby providing strong data support for subsequent model training.

[0081] In some embodiments, as shown in Figure 3 The step S230 of automatically training the original video AI algorithm model according to the preset training strategy, the training data, and the mapping relationship to obtain the target video AI algorithm model includes the following steps.

[0082] The step S231 of triggering a new round of automatic training when at least one trigger condition in the preset training strategy is met, wherein the trigger conditions in the training strategy are that the context data in the loop reaches a preset quantity threshold, the time of the last round of automatic training exceeds a preset time threshold, and a target trigger signal is received.

[0083] The step S232 of automatically training the original video AI algorithm model according to the training data and the mapping relationship to obtain the target video AI algorithm model.

[0084] Specifically, the trigger conditions in the training strategy are used as conditions for triggering a new round of automatic training, and the trigger conditions are that the context data in the loop reaches a preset quantity threshold (for example, 100, 200, 500, etc.), to prevent the training time from being uncontrollable due to too large data volume, the time of the last round of automatic training exceeds a preset time threshold (for example, 10 hours, 24 hours, 48 hours, etc.), to prevent a small amount of data from being trained for a long time, and a target trigger signal (for example, an immediate training instruction of a user) is received, to facilitate manual operation.

[0085] Further, as long as one of the above trigger conditions is met, a new round of automatic training is performed, and a new, unchangeable data set with a version number is published, that is, the original video AI algorithm model is automatically trained according to the training data and the mapping relationship to obtain the target video AI algorithm model.

[0086] In some embodiments, the step S232 of automatically training the original video AI algorithm model according to the training data and the mapping relationship to obtain the target video AI algorithm model includes the following steps.

[0087] The training data is obtained based on a preset data loading mode and a mapping relationship, and the data loading mode includes a single-process mode and a multi-process mode.

[0088] The original video AI algorithm model is loaded and corresponding training parameters are set.

[0089] The original video AI algorithm model is automatically trained according to the training data and the training parameters, and a preset key indicator is continuously monitored to automatically train the process and intelligently intervene, to obtain the target video AI algorithm model.

[0090] Specifically, the scheduler will select the relevant resources for training, such as the latest version of the training container, the training task, and the latest version of the dataset (image data after adding labels to the original training data), from the mapping relationship according to the algorithm unique identifier, and automatically start the algorithm training task. During the training process, the pre-set key indicators are continuously monitored to automatically intervene in the training process; the key indicators include but are not limited to learning rate, precision, recall, loss function value, etc.; the intelligent intervention can be: if the model performance does not improve for consecutive multiple periods, the training is terminated in advance to save resources; if the model performance continues to improve steadily but does not meet the standard, the training period is appropriately extended.

[0091] Reference Figure 4 The specific training process is described in detail as follows:

[0092] 1. Initialization phase:

[0093] 1.1, parse the incoming opt (command line parameters) and hyp (hyperparameters), create a save directory (save_dir) and weight directory (weight).

[0094] 1.2, load hyperparameters and save configuration, if the hyperparameters are strings (file paths), load the yaml file, save the hyperparameters and command line parameters to the file (hyp.yaml and opt.yaml).

[0095] 1.3, initialize log recorders (Loggers) and callback functions (Callbacks).

[0096] 1.4, load and verify the dataset configuration, use check_dataset to check the dataset, get the training and validation paths, get the number of classes (nc) and class names (names).

[0097] 2. Data preparation phase:

[0098] In the data preparation phase, the data loader (DataLoader) is used to load the training data. The data loader supports multiple data loading modes; the data loading modes include single-process mode (without GPU) and multi-process mode (with GPU).

[0099] In single-process mode, the data loader directly returns a batch of data, including input data and corresponding labels.

[0100] In multi-process mode, there are two iteration modes: recursion: sequentially from each shard (shard) to get data. Iteration mode (iter): sequentially from a shard data loader to get data, and after the current shard data loading is completed, switch to the next shard.

[0101] 3. Model building phase:

[0102] 3.1, Model loading: If the pre-training weight (ending with.pt) is specified, download and load the original video AI algorithm model and load the matching weight; otherwise, create a new original video AI algorithm model according to the cfg file.

[0103] 3.2, Set image size (imgsz) and batch size (batch_size): Check if the image size is a multiple of the model stride, and if the batch_size is -1, automatically adjust the batch size, otherwise use the specified batch size.

[0104] 3.3, Set optimizer: According to the value specified by the opt.optimizer parameter, select the optimizer (SGD, Adam, AdamW), and divide the model parameters into three groups (g0: BatchNorm weight, g1: weight with weight decay, g2: bias).

[0105] 3.4, Set learning rate scheduler: Set the cosine or linear learning rate decay according to the parameter opt.cos_lr, and set the learning rate change function lf.

[0106] 3.5, Set EMA (Exponential Moving Average): If the current process is the master process (RANK is -1 or 0), initialize the EMA model to maintain the sliding average of the model parameters, and improve the stability of the final model.

[0107] 3.6, Resume training: If the parameter resume is True, then the training needs to be resumed, then load the checkpoint (ckpt), restore the optimizer state, EMA state, start epoch (start_epoch) and best evaluation index (best_fitness).

[0108] 3.7, Distributed training settings: If using multiple GPUs and not in DDP mode (RANK is not -1 and the number of GPUs > 1), use DataParallel. If using DDP (RANK is not equal to -1), use DistributedDataParallel, and set SyncBatchNorm (if enabled).

[0109] 3.8, Create data loaders: Use create_dataloader to create training data loader (train_loader) and validation data loader (val_loader).

[0110] 3.9, Automatic anchor adjustment: If automatic anchor is not disabled (parameter noautoanchor is False), check anchors and adjust.

[0111] 3.10, Set loss function: Initialize ComputeLoss for calculating loss.

[0112] 3.11, Initialize training variables: Set warm-up iteration number (nw), loss record (mloss), evaluation metrics (maps, results), etc.

[0113] 4, Training loop phase: In the training loop, each epoch iterates through each batch in the data loader (dataloader). In batch iteration, the following steps are executed:

[0114] 4.1, Warm-up phase: In the first nw iterations of training, perform warm-up. Learning rate gradually increases from initial low value (such as hyp['warmup_bias_lr']) to initial learning rate (lr0). Momentum gradually adjusts from hyp['warmup_momentum'] to the set momentum value (hyp['momentum']). Gradient accumulation steps (accumulate) are adjusted according to the current iteration number, gradually increasing from 1 to the set accumulation steps (nbs / batch_size).

[0115] 4.2, Forward propagation: If the multi-scale parameter is enabled, randomly scale the image size (between 0.5 and 1.5 times), then adjust to the nearest grid size (gs) multiple to enhance model scale invariance. Convert image data (imgs) to tensor on device and normalize to 0-1 (divide by 255). Use mixed precision training (AMP): Perform forward propagation in the autocast context to get the prediction result (pred). Calculate loss: Use the ComputeLoss class to calculate the loss (including classification loss, positioning loss and target confidence loss), and return the total loss and each loss.

[0116] 4.3, Backpropagation: Scale the loss using scaler.scale(loss) and then perform backpropagation (backward). When the accumulated gradient steps reach accumulate, perform the optimization step (scaler.step(optimizer)), then update the scaler (scaler.update()) and clear the gradients (optimizer.zero_grad()). If EMA (Exponential Moving Average) is used, update the EMA model (ema.update(model)).

[0117] 4.4, Validation phase: At the end of each epoch, perform validation (validation condition is not disabled validation and is not the final epoch, or the final epoch) to obtain the final target video AI algorithm model. Use the EMA model to perform validation, call the val.run function to calculate mAP, and output the precision, recall, mAP@0.5, mAP@0.5:0.95, and other indicators. Update the best model weight according to the validation result (if the current fitness is better, save it as best.pt). Early stopping mechanism: set the tolerance period (e.g., patience=10) to indicate that the training will stop when the validation indicator does not improve for N consecutive periods. The validation indicator can be specified, such as precision, recall, and bounding box regression loss rate. Compare the difference between the current indicator and the historical best indicator (e.g., current_loss>best_loss+min_delta) to determine whether there is improvement or no improvement. Extend the training: set the final target indicator (e.g., finalmAP=0.8, patience=10). When the epoch is about to end, judge the difference between the validation indicator of the previous N periods and the historical best indicator. If the final target indicator is not reached, extend the training period. The number of extended training periods is proportional to the difference. In addition, logs are recorded and loss curves are drawn during training.

[0118] Through this embodiment, the whole link from data collection to model deployment is connected, realizing a closed-loop automation of "unattended", and shortening the original manual iteration period of several days or even several weeks to hours. Automatic and continuous training using real feedback data from the production environment ensures that the model can quickly adapt to changes in real scenarios and continuously improve performance. And introduce an automated scheduler to fine-mapping and version management of algorithm, data, model, service and other resources, and control the training and evaluation process through intelligent strategies, effectively saving computing resources. Further, it realizes a non-intrusive, automated and efficient data backflow, improving the practical value of video AI algorithm model.

[0119] In some embodiments, as shown in Figure 5 The automatic training method of the bytecode-enhanced video AI algorithm model further includes the following steps:

[0120] In step S510, the target video AI algorithm model is evaluated according to the unique identifier and the mapping relationship in the context data, and an evaluation result is obtained.

[0121] In step S520, when the evaluation result is evaluation passed, the corresponding target video AI algorithm model is converted according to the deployment environment, and the converted target video AI algorithm model is deployed to the target server.

[0122] Specifically, in combination with Figure 4 After the training task is completed, the scheduler will evaluate the target video AI algorithm model according to the unique identifier and the mapping relationship in the context data, and obtain an evaluation result; the evaluation includes two parts; the first part is the evaluation of the model during the training process; the second part is the evaluation of the actual performance of the model. Through these two evaluation methods, the reliability and user satisfaction of the algorithm in actual application are improved.

[0123] The first part is that the scheduler will automatically analyze all model checkpoints (Checkpoints) generated during the training process, evaluate according to the preset evaluation standard (such as the highest accuracy on the validation set, the lowest false positive rate, etc.), and obtain an evaluation result. If the optimal model file (target video AI algorithm model) can be selected, the evaluation is passed, and it is saved as a new version of the algorithm in the algorithm model library.

[0124] Among them, the evaluation standard mainly includes the following core indicators, which measure the performance of the model from multiple dimensions such as accuracy, positioning accuracy, and real-time performance.

[0125] 1. Mean Average Precision (mAP) and variant indicators, mAP (mean Average Precision) is the most core evaluation indicator, which reflects the comprehensive accuracy of the model in multi-class target detection. During the training process, the highest mAP on the validation set is usually used as the key condition for saving the optimal weight. mAP@0.5: mAP when the IOU threshold is 0.5, which is the basic standard for measuring positioning accuracy, mAP@0.5:0.95: The average mAP of the IOU threshold from 0.5 to 0.95 with an interval of 0.05, which evaluates the performance under different positioning accuracy more strictly.

[0126] 2. Validation Loss (Validation Loss) indicator, when the loss (such as classification loss, positioning loss) of the validation set reaches the minimum, the model may also be determined as the optimal, because the loss value directly reflects the fitting effect of the model on the validation data.

[0127] 3. Self-defined index combination, adjust index weight through configuration file, optimize focus according to application scene, such as lightweight model preferentially using index FPS, indicating the number of images processed per second by the model, a key indicator of real-time performance, high-precision model focuses on mAP@0.5:0.95, and comprehensive performance model can select mAP to measure, or directly determine the optimal model when the verification loss is the lowest, and finally save the model weight with the best performance for inference and deployment.

[0128] The second part is: to verify the actual performance of the target video AI algorithm model, the scheduler executes an automated evaluation process, specifically: find the corresponding online inference service instance according to the algorithm unique identifier; temporarily load the newly trained model version and perform inference tasks using the pre-set evaluation dataset; compare the inference results (precision, recall, false positive rate, etc.) of the new model with the historical published model versions, and generate an evaluation report; if the performance of the new model does not significantly improve or even decreases compared to the online version, the iteration is aborted, and the status is marked as "evaluation failed". If there is an improvement, the evaluation is passed.

[0129] After the evaluation is passed, model conversion and image building are performed; specifically:

[0130] For the target video AI algorithm model that passes the evaluation, the scheduler calls the model conversion service to convert it from the format used during training (such as PyTorch or TensorFlow format) to an executable inference format (such as TensorRT, ONNX, etc.) optimized for production environment to improve inference speed. The converted model is packaged into a new algorithm image with version number.

[0131] Finally, model hot update and deployment are performed, specifically: the scheduler queries the list of servers running this algorithm in the production environment according to the algorithm unique identifier, pushes the newly built algorithm image to these servers, and issues instructions to smoothly switch the online service to the new algorithm image version. Thus, the model is updated and deployed.

[0132] Through this embodiment, through automated closed-loop iteration and strict evaluation process, the accuracy and recall rate of the model can be continuously optimized, the false positive rate can be reduced, and the reliability and user satisfaction of the algorithm in actual application can be improved.

[0133] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0134] An apparatus for automatically training a bytecode enhancement-based video AI algorithm model is also provided in this embodiment. The apparatus is used to implement the above-described embodiments and preferred embodiments, and has been described above. The terms "module", "unit", "sub-unit", and the like used below can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware can also be implemented and conceived.

[0135] Figure 6 is a structural block diagram of the bytecode enhancement-based video AI algorithm model automatic training apparatus of this embodiment, as Figure 6 shown, the apparatus includes a construction module 210, an acquisition and annotation module 220, and a training module 230.

[0136] The construction module 210 is configured to construct a data backflow agent based on a preset bytecode enhancement proxy mechanism and a bytecode enhancement library, and transmit context data dynamically intercepted to a dispatcher based on the data backflow agent. The bytecode enhancement proxy mechanism has a plug-in architecture for isolated loading with a target application. The dispatcher is configured with a mapping relationship between algorithms, resources, and tasks.

[0137] The acquisition and annotation module 220 is configured to acquire corresponding image data in the dispatcher according to the context data and the mapping relationship, and intelligently annotate the image data to obtain training data.

[0138] The training module 230 is configured to automatically train an original video AI algorithm model according to a preset training strategy, training data, and the mapping relationship to obtain a target video AI algorithm model.

[0139] The above-described apparatus solves the problem in the related art that the data backflow link is long, resulting in low efficiency, strong application invasiveness, high development cost, and difficulty in maintenance and upgrading. The plug-in architecture in the bytecode enhancement proxy mechanism is used to isolate the target application, implement non-invasive data backflow, reduce development cost, facilitate maintenance and upgrading, and automatically dynamically intercept context data, thereby reducing the data backflow link and improving efficiency.

[0140] In some embodiments, the bytecode enhancement proxy mechanism further includes:

[0141] The data backflow agent is loaded through a virtual machine startup parameter or an environment variable. The dynamic bytecode enhancement strategy is used. The asynchronous data reporting strategy is used.

[0142] In some embodiments, the asynchronous data reporting strategy includes:

[0143] The context data is transmitted to the scheduler in a non-blocking I / O or multi-thread concurrent processing manner.

[0144] In some embodiments, the labeling module 220 is further configured to invoke an intelligent labeling service to perform intelligent labeling on the image data.

[0145] The image data marked as error samples is recorded in a non-target example library.

[0146] The image data marked as correct samples is recorded in a target example library to obtain training data.

[0147] In some embodiments, the training module 230 is further configured to trigger a new round of automatic training when at least one trigger condition in a preset training strategy is met, wherein the trigger conditions in the training strategy are that the backflow context data reaches a preset quantity threshold, the time of the last round of automatic training exceeds a preset time threshold, and a target trigger signal is received.

[0148] The original video AI algorithm model is automatically trained based on the training data and the mapping relationship to obtain a target video AI algorithm model.

[0149] In some embodiments, the training module 230 is further configured to obtain the training data based on a preset data loading mode and the mapping relationship, wherein the data loading mode includes a single-process mode and a multi-process mode.

[0150] The original video AI algorithm model is loaded and corresponding training parameters are set.

[0151] The original video AI algorithm model is automatically trained based on the training data and the training parameters, and a preset key indicator is continuously monitored to intelligently intervene in the automatic training process to obtain a target video AI algorithm model.

[0152] In some embodiments, the automatic training device of the video AI algorithm model based on bytecode enhancement further comprises an evaluation and deployment module.

[0153] The evaluation and deployment module is configured to evaluate the target video AI algorithm model based on the unique identifier in the context data and the mapping relationship to obtain an evaluation result.

[0154] When the evaluation result is passed, the corresponding target video AI algorithm model is converted according to a deployment environment, and the converted target video AI algorithm model is deployed to a target server.

[0155] It should be noted that the above various modules can be functional modules or program modules, which can be implemented by software or hardware. For the modules implemented by hardware, the above various modules can be located in the same processor; or the above various modules can also be located in different processors in any combination.

[0156] In the embodiment, a computer device is also provided, including a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the above method embodiments.

[0157] Optionally, the computer device can further include a transmission device and an input and output device, wherein the transmission device is connected with the processor, and the input and output device is connected with the processor.

[0158] Optionally, in the embodiment, the processor can be configured to execute the following steps through the computer program:

[0159] S1, based on the preset bytecode enhancement agent mechanism and bytecode enhancement library, a data backflow agent is constructed, and the context data dynamically intercepted is transmitted to a scheduler based on the data backflow agent; the bytecode enhancement agent mechanism has a plug-in architecture, and the plug-in architecture is used for isolated loading with a target application; the scheduler is configured with an algorithm respectively mapping a resource and a task;

[0160] S2, in the scheduler, corresponding image data is obtained according to the context data and the mapping relationship; and the image data is intelligently labeled to obtain training data;

[0161] S3, according to a preset training strategy, training data and the mapping relationship, an original video AI algorithm model is automatically trained to obtain a target video AI algorithm model.

[0162] It should be noted that the specific examples in the embodiment can refer to the examples described in the above embodiments and optional implementation manners, which will not be described herein.

[0163] In addition, in combination with the automatic training method of the video AI algorithm model based on bytecode enhancement provided in the above embodiments, a storage medium can also be provided to implement the automatic training method of the video AI algorithm model based on bytecode enhancement in the embodiment. The storage medium stores a computer program; the computer program is executed by a processor to implement any of the automatic training methods of the video AI algorithm model based on bytecode enhancement in the above embodiments.

[0164] It should be noted that the information and data involved in the present application are all authorized information and data or information and data authorized by all parties, which will be used legally.

[0165] It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not to be taken in a limiting sense. All other embodiments falling within the scope of the application are contemplated and are within the scope of the application.

[0166] It is apparent that the drawings depicted are only a few examples of the application and that many other applications can be derived from these using the concepts disclosed in this application without applying inventive skills. Therefore, the scope of the application should be determined by the appended claims.

[0167] The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations. The specific exemplary embodiments described herein are presented for purposes of clarity and / or enabling a complete description of the application. Any

[0168] The above-described embodiments are merely illustrative of several ways to make and use the application and do not delimit the scope of the patent protection. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the application, and these are within the scope of the patent protection. Therefore, the scope of the patent protection should be determined by the appended claims.

Claims

1. An automatic training method of a video AI algorithm model based on bytecode enhancement, characterized in that, The method comprises the following steps: Based on the preset bytecode enhancement agent mechanism and bytecode enhancement library, a data backflow agent is constructed, and context data dynamically intercepted is transmitted to a scheduler based on the data backflow agent; The bytecode enhancement agent mechanism has a plug-in architecture for isolated loading with a target application; the scheduler is configured with an algorithm mapping relationship with resources and tasks; wherein the resources include image data; In the scheduler, corresponding image data is obtained according to the context data and the mapping relationship; and the image data is intelligently annotated to obtain training data; wherein the context data includes the unique identifier of the algorithm; According to a preset training strategy, the training data and the mapping relationship, an original video AI algorithm model is automatically trained to obtain a target video AI algorithm model.

2. The automatic training method of a bytecode enhancement-based video AI algorithm model according to claim 1, characterized in that, The bytecode enhancement agent mechanism further comprises: The data backflow agent is loaded through the startup parameters or environment variables of the virtual machine; a dynamic bytecode enhancement strategy; and an asynchronous data reporting strategy.

3. The automatic training method of a video AI algorithm model based on bytecode enhancement according to claim 2, characterized in that, The asynchronous data reporting strategy comprises: The context data is transmitted to the scheduler in a non-blocking I / O or multi-thread concurrent processing manner.

4. The automatic training method of a video AI algorithm model based on bytecode enhancement according to claim 1, characterized in that, Intelligently annotating the image data to obtain training data comprises: Calling an intelligent annotation service to intelligently annotate the image data; Recording the image data marked as error samples in a non-target example library; Recording the image data marked as correct samples in a target example library to obtain training data.

5. The automatic training method of a bytecode enhancement-based video AI algorithm model according to any one of claims 1 to 4, characterized in that, According to a preset training strategy, the training data and the mapping relationship, an original video AI algorithm model is automatically trained to obtain a target video AI algorithm model, comprising: When at least one trigger condition in the preset training strategy is met, a new round of automatic training is triggered; the trigger conditions in the training strategy are that the context data backflow reaches a preset quantity threshold; the time of the last round of automatic training exceeds a preset time threshold; and a target trigger signal is received; According to the training data and the mapping relationship, the original video AI algorithm model is automatically trained to obtain the target video AI algorithm model.

6. The automatic training method of a video AI algorithm model based on bytecode enhancement according to claim 4, characterized in that, According to the training data and the mapping relationship, the original video AI algorithm model is automatically trained to obtain the target video AI algorithm model, comprising: Based on a preset data loading mode and the mapping relationship, the training data is obtained; the data loading mode includes a single-process mode and a multi-process mode; Loading the original video AI algorithm model and setting corresponding training parameters; According to the training data and the training parameters, the original video AI algorithm model is automatically trained, and the preset key indicators in the automatic training process are continuously monitored for intelligent intervention to obtain the target video AI algorithm model.

7. The automatic training method of a video AI algorithm model based on bytecode enhancement according to claim 4, characterized in that, The method further comprises: According to the unique identifier in the context data and the mapping relationship, the target video AI algorithm model is evaluated to obtain an evaluation result; When the evaluation result is evaluation pass, the corresponding target video AI algorithm model is converted according to a deployment environment, and the converted target video AI algorithm model is deployed to a target server.

8. An automatic training device of a video AI algorithm model based on bytecode enhancement, characterized in that, Comprise: The construction module, the acquisition annotation module and the training module; The construction module is used for constructing a data backflow agent based on a preset bytecode enhancement agent mechanism and a bytecode enhancement library, and transmitting context data dynamically intercepted to a dispatcher based on the data backflow agent; The bytecode enhancement agent mechanism has a plug-in architecture for isolated loading with a target application; the dispatcher is configured with a mapping relationship between algorithms, resources and tasks; wherein the resources include image data; The acquisition annotation module is used for acquiring corresponding image data in the dispatcher according to the context data and the mapping relationship, and intelligently annotating the image data to obtain training data; wherein the context data includes a unique identifier of an algorithm; The training module is used for automatically training an original video AI algorithm model according to a preset training strategy, the training data and the mapping relationship to obtain a target video AI algorithm model. 9.A computer device, comprising a memory and a processor, and characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute the steps of the automatic training method of the video AI algorithm model based on bytecode enhancement in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the automatic training method of the video AI algorithm model based on bytecode enhancement in any one of claims 1 to 7.

Images