Dynamic prediction of labeling task workload and resource allocation method

Abstract

The application discloses a dynamic prediction and resource allocation method for labeling task workload, and belongs to the technical field of resource scheduling, comprising: acquiring labeling task data, and generating a key feature vector by using a feature extraction model; determining the core attributes and difficulty levels of the task by a full connection layer classifier, and dividing coarse and fine granularity workload dimensions containing macro and micro indicators according to the core attributes and difficulty levels; in the prediction stage, a double model collaborative mechanism is adopted to dynamically correct and output the final total task time consumption and time consumption distribution; the prediction result is analyzed, and the Hungarian algorithm and a priority arbitration mechanism are combined to automatically optimally match labeling personnel, tool instances and time windows for task sub-items, so that precise and efficient resource automation allocation is realized; and the method improves the intelligent level of labeling project management and the resource utilization efficiency.

Description

Technical Field

[0001] This invention belongs to the field of resource scheduling technology, specifically a method for dynamic prediction and resource allocation of labeled task workload. Background Technology

[0002] In the training process of artificial intelligence models, data annotation is a core preliminary step. The accuracy of workload prediction and the rationality of resource allocation for annotation tasks directly affect project progress and annotation quality. Traditional annotation work relies on manual workload estimation, which suffers from problems such as large prediction errors, rigid resource allocation, and untimely response. With the expansion of annotation task scale and the diversification of data types, there is an urgent need for intelligent methods to achieve accurate workload prediction and dynamic resource scheduling. Existing technologies are often limited to single-dimensional workload estimation or fixed-rule resource allocation, making it difficult to adapt to complex and ever-changing annotation task scenarios, such as different data types, annotation difficulty, and deadline requirements. They suffer from weak generalization ability, poor real-time performance, and low resource utilization, failing to meet the needs of efficient management of large-scale, multi-type annotation tasks. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes a dynamic prediction and resource allocation method for annotation task workload. The method acquires annotation task data and generates key feature vectors using a feature extraction model. A fully connected layer classifier determines the core attributes and difficulty level of the task, and accordingly divides the workload into coarse and fine-grained dimensions, including macro and micro indicators. During the prediction phase, a dual-model collaborative mechanism dynamically corrects and outputs the final total task time and time distribution. The prediction results are analyzed, and combined with the Hungarian algorithm and a priority arbitration mechanism, the method automatically and optimally matches annotation personnel, tool instances, and time windows for each task sub-item, thereby achieving precise and efficient automated resource allocation. This method improves the intelligence level and resource utilization efficiency of annotation project management.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] Methods for dynamic prediction and resource allocation of workload for annotation tasks include:

[0006] Acquire annotation task data; the annotation task data contains multiple key features of the task.

[0007] Based on the extracted key features of the task, the core attributes and difficulty level of the labeled task are determined.

[0008] Based on the established core attributes and difficulty levels, the annotation task is divided into coarse-grained workload dimension and fine-grained workload dimension;

[0009] Based on coarse-grained and fine-grained workload dimensions, an AI model is used to dynamically predict the workload of the annotation task, and the predicted total task time and time distribution are output.

[0010] Based on the dynamic prediction results, annotation personnel, tools, and time resources are automatically allocated.

[0011] Specifically, the process of determining the core attributes and difficulty level of the labeled task based on the extracted key features includes:

[0012] The labeled task data is input into the feature extraction model, which outputs a key feature vector for the task. The key feature vector includes the data size quantization value, data type encoding, label type encoding, single sample information density value, and historical average labeling efficiency. The feature extraction model consists of three layers of convolutional neural network and two layers of long short-term memory network connected in sequence. The convolutional neural network is used to extract local numerical features in the data size and single sample information density, while the long short-term memory network is used to learn the label type and the sequence dependency relationship in the historical labeled data.

[0013] The key feature vectors of the task are input into a fully connected layer classifier, which outputs core attributes and difficulty level. The core attributes include text annotation, image annotation, speech annotation, single-label annotation, and multi-label annotation, which are jointly determined by the data type encoding and the annotation type encoding. The difficulty level includes low, medium, and high difficulty levels, which are calculated by the fully connected layer classifier based on the data scale quantization value, single sample information density value, and historical average annotation efficiency.

[0014] Specifically, in the feature extraction model, the three-layer convolutional neural network is configured as follows: the first convolutional layer uses 64 filters of size 3×1 with a stride of 1, and the output feature map is downsampled by a max pooling layer; the second convolutional layer uses 128 filters of size 3×1; the third convolutional layer uses 256 filters of size 3×1; the two subsequent long short-term memory (LSTM) networks have 128 and 64 hidden units respectively, and the final hidden state of the last LSM network serves as a sequence information summary of the task's key feature vectors.

[0015] Specifically, in the feature extraction model, the specific rules for calculating the difficulty level of the fully connected layer classifier are as follows:

[0016] The initial difficulty score is obtained by multiplying the quantified value of the data size by the information density value of a single sample.

[0017] If there are historical annotation tasks that match the current task to a preset matching threshold and the historical average annotation efficiency is lower than the system benchmark efficiency, then the initial difficulty score is weighted and increased according to the degree to which it is lower than the system benchmark efficiency to obtain the increased difficulty score.

[0018] Based on the range of the increased difficulty score, it is mapped to low, medium, and high difficulty levels.

[0019] Specifically, the process of dividing the coarse-grained workload dimension and the fine-grained workload dimension includes:

[0020] Based on the core attributes and difficulty level output by the fully connected layer classifier, the core dimension index set for workload prediction is queried and determined from a pre-configured dimension index mapping table; the core dimension index set includes a macro-index subset for coarse-grained prediction and a micro-index subset for fine-grained prediction.

[0021] A coarse-grained workload dimension is constructed based on the aforementioned subset of macro indicators; the coarse-grained workload dimension includes the overall data volume of the task, a unit data volume weight coefficient based on core attributes, and a baseline efficiency adjustment coefficient calculated based on the difficulty level and historical average annotation efficiency;

[0022] A fine-grained workload dimension is constructed based on the subset of micro-indicators; the fine-grained workload dimension includes the basic time consumption per sample calculated based on the single sample information density value and the annotation type encoding, the data complexity multiplier determined based on the difficulty level, and the accuracy correction coefficient derived from the annotation accuracy requirements.

[0023] Specifically, the dynamic prediction of the workload of the annotation task using an AI model, and the output of the predicted total task time and time distribution, includes:

[0024] Based on the coarse-grained workload dimension, the preprocessed labeled task data is used as a coarse-grained task sample set and input into the first deep learning model. The first deep learning model is a network based on gated recurrent units. Its input layer receives the macro-indicator subset, and its output layer outputs the coarse-grained workload prediction value, including the total estimated task hours and the total manpower requirement.

[0025] While the first deep learning model is performing calculations, its hidden state vector is captured in real time and input into a dynamic splitting module; the dynamic splitting module includes a complexity analysis unit, which is used to calculate the data complexity coefficient and annotation difficulty fluctuation value of each data block in the coarse-grained task sample set.

[0026] When the complexity analysis unit identifies that the data complexity coefficient or annotation difficulty fluctuation value of any data block exceeds a preset threshold, it triggers a sample set splitting operation to extract the corresponding data block from the coarse-grained task sample set to form a fine-grained task sample set.

[0027] The fine-grained task sample set and its corresponding micro-indicator subset are input into the second deep learning model; the second deep learning model is a multilayer perceptron network, whose input layer is connected to the fully connected layer of the micro-indicator subset, the hidden layer uses the ReLU activation function, and the output layer outputs the fine-grained workload prediction value through the Sigmoid function, including the average time per sample, the proportion of data at different difficulty levels, and the time fluctuation range;

[0028] The output of the second deep learning model is fed back to the gated recurrent unit of the first deep learning model to correct the coarse-grained workload prediction value, so as to obtain the final dynamically predicted total task time and time distribution.

[0029] Specifically, the output of the second deep learning model is fed back to the gated recurrent unit of the first deep learning model, and the process of correcting the coarse-grained workload prediction includes:

[0030] The average time and time fluctuation range of a single sample in the fine-grained workload prediction value are converted into an efficiency correction factor; the efficiency correction factor is multiplied by the hidden state vector of the first deep learning model to adjust its memory and prediction of the total task time.

[0031] Specifically, the automatic allocation of annotation personnel, tools, and time resources based on dynamic prediction results includes:

[0032] The total task time and time distribution of the dynamic prediction results are analyzed to generate a resource requirement list.

[0033] The resource requirement list is matched with the real-time resource pool, which dynamically updates the current status of the annotators, the tool load, and the time slot information.

[0034] The optimal matching calculation is performed based on the Hungarian algorithm, and annotation personnel, idle annotation tool instances and continuous time windows are allocated to each annotation task sub-item, and a resource allocation plan is generated.

[0035] The resource allocation scheme is executed, and the allocation results are compared with the actual annotation progress data. The difference between the comparisons is used as feedback data to adjust the weight parameters of the first deep learning model and the second deep learning model.

[0036] Specifically, when performing optimal matching calculations based on the Hungarian algorithm, if multiple labeling task sub-items compete for the same resource, a priority arbitration mechanism is set up, including: prioritizing the allocation of labeling task sub-items with high difficulty level, with priority order being high difficulty, medium difficulty, and low difficulty; if multiple labeling task sub-items have the same difficulty level, prioritizing the allocation of the labeling task sub-item with the shortest remaining available labeling time, where the remaining available labeling time is the difference between the task deadline and the current system time; for task sub-items that do not obtain the corresponding resource due to conflict, they automatically enter the next round of matching queue within the allowed time window.

[0037] Specifically, the allocation of the annotation tool instances is for the target detection task in image annotation: a GPU computing power instance equipped with a pre-trained model is allocated to the first annotation task sub-item; a standard CPU computing power instance is allocated to the second annotation task sub-item; the allocation information and annotation task data are sent to the client together, and the allocated tool environment is automatically loaded.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] 1. This invention proposes a dynamic prediction and resource allocation method for annotation task workload. This invention uses an AI model to accurately predict and decompose the workload of annotation tasks, resulting in a significant improvement in management efficiency. The method first uses deep learning technology to automatically analyze task characteristics, accurately divide the workload into coarse and fine granular dimensions, and adopts a dual-model collaborative prediction mechanism. While outputting the total task time, it can also provide a detailed time distribution, making workload estimation change from static and experience-based to dynamic and data-driven, which greatly improves the accuracy of prediction and adaptability to complex tasks.

[0040] 2. This invention proposes a dynamic prediction and resource allocation method for annotation task workload. Based on accurate prediction, this method achieves automated and optimized resource allocation. The system can intelligently match and allocate annotation personnel with corresponding skills, tool instances with different computing power, and reasonable time windows according to the dynamic prediction results. By introducing the Hungarian algorithm and priority arbitration mechanism, the problem of multi-task resource competition is effectively solved, thereby maximizing resource utilization, shortening the project cycle, and continuously optimizing the model through feedback on actual progress, ultimately ensuring that the annotation task is completed efficiently and with high quality. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the dynamic prediction and resource allocation method for labeled task workload of the present invention;

[0042] Figure 2 This is a flowchart illustrating the principle of the dynamic prediction and resource allocation method for labeled task workload in this invention.

[0043] Figure 3 This is a flowchart illustrating the dynamic prediction and resource allocation method for task workload of the present invention, which predicts the total task time and time distribution. Detailed Implementation

[0044] Example 1:

[0045] Please see Figure 1 and Figure 2 The present invention provides an embodiment of a method for dynamic prediction and resource allocation of labeled task workload, the method comprising S1 to S5, including the following steps:

[0046] S1: Obtain annotation task data; the annotation task data contains multiple key features of the task;

[0047] Furthermore, the specific process of obtaining annotation task data includes: receiving newly submitted annotation task request packages from the project management system or through a dedicated data interface; the annotation task request package is usually encapsulated in a structured format such as JSON or XML, containing information such as the basic description of the task, the original data storage path, the annotation specification document link, and the expected deadline; the system parses the annotation task request package and accesses the specified data storage location, such as the company's internal NAS network attached storage or cloud storage services such as AWS S3 bucket, to read the original data files to be annotated in batches. At the same time, the system queries the historical task database to retrieve records of historical tasks that are similar to the current task in terms of data type and annotation type, so as to obtain historical efficiency data for reference.

[0048] This embodiment takes a specific task of multi-target detection and attribute annotation in urban street view images as an example. The task aims to annotate 100,000 urban road images with a resolution of 1920×1080 pixels, captured by vehicle-mounted cameras. It requires not only outlining all targets such as vehicles, pedestrians, and traffic signs in the images, but also annotating the specific attributes of each target, such as the color and model of vehicles, the posture of pedestrians, and the specific meaning of traffic signs. This is a typical image multi-label annotation task with a huge data scale, high information density of single images, complex scenes, and numerous targets, requiring extremely high annotation accuracy. The system obtained the storage index of these 100,000 images, detailed annotation specification documents, and the four-week completion deadline required by the project. By querying the historical database, it was found that a single-target detection task of 50,000 similar street view images completed half a year ago had a historical average annotation efficiency of about 50 images per hour.

[0049] S2: Based on the extracted key features of the task, determine the core attributes and difficulty level of the labeled task;

[0050] S3: Based on the determined core attributes and difficulty levels, the annotation task is divided into coarse-grained workload dimension and fine-grained workload dimension;

[0051] S4: Based on coarse-grained and fine-grained workload dimensions, use AI models to dynamically predict the workload of annotation tasks, and output the predicted total task time and time distribution.

[0052] S5: Automatically allocate annotation personnel, tools, and time resources based on dynamic prediction results.

[0053] Furthermore, this method is applicable to situations where multiple independent labeled projects are processed simultaneously. The system maintains a global resource view. When performing resource matching, the input of the Hungarian algorithm is the resource demand matrix of all task sub-items in all projects and the supply matrix of the global resource pool, so as to achieve global optimal resource allocation across projects and avoid resource silos.

[0054] Example 2:

[0055] Please see Figure 3 In this embodiment, the process of determining the core attributes and difficulty level of the labeled task based on the extracted key features includes:

[0056] S2.1: Input the annotation task data into the feature extraction model, and the feature extraction model outputs the task key feature vector; the task key feature vector includes the data size quantization value, data type encoding, annotation type encoding, single sample information density value, and historical average annotation efficiency; the feature extraction model consists of three layers of convolutional neural network and two layers of long short-term memory network connected in sequence, wherein the convolutional neural network is used to extract local numerical features in data size and single sample information density, and the long short-term memory network is used to learn the sequence dependency relationship in annotation type and historical annotation association data;

[0057] In this embodiment, numerical and categorical features in the task data are converted into tensor formats that the model can process. For example, the data size of "100,000 images" is quantized as the value 100,000; the data type "image" is encoded with predefined category codes, such as 1 for image, 2 for text, and 3 for speech; the annotation type "multi-label" is also encoded similarly; the average number of targets per image is used as the single-sample information density value; and the historical average annotation efficiency of "50 images / hour" is used as a reference value input.

[0058] In the feature extraction model, the specific configuration of the three-layer convolutional neural network is as follows: The first convolutional layer uses 64 filters of size 3×1 with a stride of 1, aiming to capture local fluctuation patterns in these numerical features. For example, when the data scale is at the level of hundreds of thousands, the corresponding resource requirements are not a simple linear relationship. The convolutional layer can learn the local features of this non-linear relationship. After convolution, a max pooling layer is used for downsampling to retain significant features; the second convolutional layer uses 128 filters of size 3×1 to further extract more abstract feature combinations; the third convolutional layer uses 256 filters of size 3×1 to output a high-level numerical feature representation.

[0059] In the feature extraction model, the two-layer long short-term memory network processing includes: the feature sequence processed by the convolutional neural network, together with the encoded category features, such as the label type encoding, and sequence dependency information such as historical average label efficiency, are input into the two-layer long short-term memory network LSTM. The first layer of LSTM has 128 hidden units, and the second layer of LSTM has 64 hidden units. LSTM networks are good at learning long-term dependencies in sequences. In this embodiment, it is used to learn sequence patterns such as changes in label type, historical efficiency trends, such as changes from single label to multi-label, and changes in efficiency of similar tasks over time. The final hidden state of the last layer of LSTM gathers a summary of all these feature sequence information, forming a fixed-length feature vector rich in contextual information.

[0060] Furthermore, the outputs of the convolutional neural network and the long short-term memory network are weighted and summed to form a task key feature vector. The task key feature vector is a low-dimensional, dense real number vector that comprehensively represents key information such as data scale quantization value, data type encoding, annotation type encoding, single sample information density value, and historical average annotation efficiency.

[0061] S2.2: Input the key feature vector of the task into a fully connected layer classifier, and the fully connected layer classifier outputs the core attributes and difficulty level;

[0062] The core attributes include text annotation, image annotation, speech annotation, single-label annotation, and multi-label annotation, which are jointly determined by the data type encoding and the annotation type encoding. The fully connected layer classifier maps the corresponding encoding information in the input task key feature vector to the predefined core attribute category. In this embodiment, the data type is image and the annotation type is multi-label; therefore, the fully connected layer classifier outputs the core attributes of the task as image annotation and multi-label annotation.

[0063] The difficulty levels include low, medium, and high difficulty, which are calculated by the fully connected layer classifier based on the data scale quantization value, single sample information density value, and historical average annotation efficiency.

[0064] In the feature extraction model, the specific rules for calculating the difficulty level of the fully connected layer classifier are as follows:

[0065] S2.2.1: Multiply the data size quantification value by the single sample information density value to obtain an initial difficulty score reflecting the overall work complexity: 100000×15=1500000, where 100000 refers to the data size quantification value and 15 refers to the single sample information density value, reflecting the basic logic that the larger the data volume and the more complex the single sample, the higher the overall difficulty.

[0066] S2.2.2: Query historical annotation data. If there are historical annotation tasks whose key features, such as core attributes, data types, annotation types, and single-sample information density values, match the current task to a preset matching threshold, and the historical average annotation efficiency is lower than the system benchmark efficiency, then the initial difficulty score is weighted and increased according to the degree of lower efficiency than the system benchmark efficiency to obtain an increased difficulty score. For example, if a similar task is found, such as street view images, but its historical average annotation efficiency is for single-label tasks, where the historical average annotation efficiency is 50 images / hour, while the current task requires higher multi-label annotation, the system presets a benchmark efficiency. For example, for medium-complexity image single-label annotation, the benchmark efficiency may be set to 60 images / hour. The historical reference efficiency of the current task is lower than the benchmark efficiency, and the current task is more complex. Therefore, the fully connected layer classifier will weight and increase the initial difficulty score according to the degree of lower efficiency than the benchmark efficiency, for example, 20% lower, for example, 30%, to obtain an increased difficulty score of: 1,500,000 × 1.3 = 1,950,000.

[0067] S2.2.3: Based on the range of the improved difficulty score, it is mapped to low difficulty, medium difficulty, and high difficulty levels. For example, a score below 500,000 is low difficulty, 500,000 to 1,500,000 is medium difficulty, and a score above 1,500,000 is high difficulty. In this embodiment, the improved difficulty score of the task is 1,950,000, which far exceeds the threshold of 1,500,000. Therefore, it is judged as high difficulty level by the fully connected layer classifier.

[0068] The process of dividing the coarse-grained workload dimension and the fine-grained workload dimension includes:

[0069] S3.1: Based on the core attributes and difficulty level output by the fully connected layer classifier, query and determine the core dimension index set for workload prediction from a pre-configured dimension index mapping table; the core dimension index set includes a macro index subset for coarse-grained prediction and a micro index subset for fine-grained prediction.

[0070] S3.2: Construct a coarse-grained workload dimension based on the aforementioned macro-indicator subset; the coarse-grained workload dimension includes the overall data volume of the task, the unit data volume weight coefficient based on core attributes, and the baseline efficiency adjustment coefficient calculated based on the difficulty level and historical average annotation efficiency;

[0071] In this embodiment, the coarse-grained dimension focuses on the overall macro-level indicators of the task. According to the dimension indicator mapping table, the subset of macro-level indicators used for coarse-grained prediction includes: the total amount of data for the task, such as 100,000 images; the unit data volume weight coefficient based on core attributes, for example, the basic weight coefficient for image annotation is set to 1.0, for text annotation to 0.5, and for voice annotation to 1.2; and the baseline efficiency adjustment coefficient calculated based on the difficulty level and the historical average annotation efficiency, for example, high difficulty may lead to a 30% reduction in efficiency, i.e., the adjustment coefficient is 0.7.

[0072] Furthermore, the process of constructing the coarse-grained workload dimension includes:

[0073] (1) The system receives a subset of macro indicators, which includes the total data volume of the task, the core attributes of the task, the difficulty level of the task, and the historical average annotation efficiency. The core attributes of the task are used to query the built-in core attribute weight dictionary to obtain the corresponding benchmark weight value. The core attribute weight dictionary defines the inherent complexity differences of different attributes, such as text, image, and speech annotation. The historical average annotation efficiency and the difficulty level of the task are temporarily stored.

[0074] (2) The system fine-tunes the baseline weight value according to the specific requirements of the task, such as professional knowledge and special annotation specifications, and generates the final unit data volume weight coefficient. The unit data volume weight coefficient reflects the unit data workload density after calibration.

[0075] (3) Input the task difficulty level into the difficulty-efficiency impact mapping table and query to obtain the difficulty adjustment multiplier; the difficulty-efficiency impact mapping table defines the quantitative impact of different difficulties on work efficiency;

[0076] (4) Multiply the historical average annotation efficiency by the difficulty adjustment multiplier to obtain the expected efficiency value;

[0077] (5) Compare the expected efficiency value with the standard efficiency value set by the system, and calculate the baseline efficiency adjustment coefficient; the baseline efficiency adjustment coefficient represents the relative efficiency level under the current task conditions;

[0078] (6) The system combines all the outputs of (1)-(5) into a complete coarse-grained workload dimension, including the total data volume of the task, the weight coefficient of the unit data volume, and the baseline efficiency adjustment coefficient;

[0079] (7) Output the synthesized coarse-grained workload dimension to the first deep learning model for global workload prediction;

[0080] (8) The system records the key parameters of this construction process, including the baseline weight value used, the difficulty adjustment multiplier, and the baseline efficiency adjustment coefficient;

[0081] (9) When the actual labeled data of this task is generated, the system will compare the predicted macro workload with the actual workload to generate deviation data. This deviation data is fed back to the core attribute weight dictionary and the difficulty-efficiency impact mapping table for adaptive optimization and update of its internal parameters.

[0082] S3.3: Construct a fine-grained workload dimension based on the subset of micro-indicators; the fine-grained workload dimension includes the basic time consumption per sample calculated based on the single sample information density value and the annotation type encoding, the data complexity multiplier determined based on the difficulty level, and the accuracy correction coefficient derived from the annotation accuracy requirements.

[0083] In this embodiment, the fine-grained dimension delves into the task itself, focusing on micro-level fluctuations. Based on the dimension index mapping table, the subset of micro-indicators used for fine-grained prediction includes: the basic time consumption per sample calculated based on the single-sample information density value and annotation type encoding, where the single-sample information density value is set to 15 targets / image. For example, it takes an average of 10 seconds to box a target and 5 seconds to annotate an attribute, so the basic time consumption per sample = 15 × 10 + 15 × 5 = 225 seconds; a data complexity multiplier determined based on the difficulty level, for example, a multiplier of 1.5 for high difficulty, as images may contain challenges such as occlusion, lighting variations, and small targets; and a precision correction coefficient derived from the annotation accuracy requirements. According to the specification document, this embodiment requires above 99% accuracy; for example, high precision requirements may require an additional 10% of the inspection time, i.e., a coefficient of 1.1. These indicators are used to predict the time consumption and its fluctuations at the single-sample level.

[0084] Furthermore, the process of constructing the fine-grained workload dimension includes:

[0085] (1) The system receives a subset of micro-indicators, which includes the annotation type code, the original value of single sample information density, the task difficulty level and the annotation accuracy requirements.

[0086] (2) Based on the annotation type code, query the built-in annotation type knowledge base to obtain the basic time consumption base corresponding to the annotation type code. The basic time consumption base represents the time base required to process a standard sample under an ideal state.

[0087] (3) Input the original value of the single sample information density into a density standardization module. The density standardization module maps it to a uniform standard density score between 0.5 and 5 according to the data type to eliminate the influence of the dimension.

[0088] (4) Input the label type code and standard density score into a nonlinear mapping function. The nonlinear mapping function uses the label type code as an index to select a preset linear growth curve and calculates the uncalibrated time of a single sample based on the standard density score. The uncalibrated time of a single sample only reflects the basic workload of the sample's intrinsic attributes and the labeling operation itself.

[0089] (5) Input the label type code and task difficulty level into the difficulty-complexity mapping table, and query to obtain the corresponding data complexity multiplier. The data complexity multiplier represents the workload amplification effect brought about by the overall increase in task difficulty.

[0090] (6) Multiply the output single-sample uncalibrated time by the data complexity multiplier obtained in this query to get the time after difficulty correction;

[0091] (7) Input the annotation accuracy requirement into an accuracy-efficiency relationship model. The accuracy-efficiency relationship model outputs an accuracy correction coefficient based on the marginal increasing effect of the annotation accuracy requirement.

[0092] (8) Multiply the output time after difficulty correction by the accuracy correction coefficient obtained in this calculation, and output the final calibrated single-sample base time;

[0093] (9) The system synthesizes a fine-grained workload dimension, which takes the calibrated single-sample basic time consumption as the core and includes the data complexity multiplier and accuracy correction coefficient as auxiliary features, and outputs them together to the second deep learning model, namely the multilayer perceptron network used to process the fine-grained task sample set.

[0094] (10) The system records the key parameters of this construction process, including the characteristic curve identifier of the nonlinear mapping function used, the data complexity multiplier and the accuracy correction coefficient. When the actual completion data of this annotation task is generated, the system compares the calibrated single sample base time with the actual annotation time to generate deviation data. This deviation data is fed back to the annotation type knowledge base, the difficulty-complexity mapping table and the accuracy-efficiency relationship model to drive the internal parameters to be updated adaptively.

[0095] The method of dynamically predicting the workload of labeling tasks using AI models and outputting the predicted total task time and time distribution includes:

[0096] S4.1: Based on the coarse-grained workload dimension, the preprocessed labeled task data is used as a coarse-grained task sample set and input into the first deep learning model; the first deep learning model is a network based on gated recurrent units, whose input layer receives the macro-indicator subset and whose output layer outputs the coarse-grained workload prediction value, including the total estimated task time and the total manpower requirement.

[0097] In this embodiment, the input layer of the network based on the gated recurrent unit receives a subset of macro-indicators, including the total data volume of the task, the weight coefficient of the unit data volume, and the baseline efficiency adjustment coefficient. Internally, through the gating mechanism of the gated recurrent unit, such as the reset gate and the update gate, it learns the complex mapping relationship between the macro-indicators and the total working hours of the historical task. After passing through the network based on the gated recurrent unit, the output layer outputs a coarse-grained workload prediction value, including the total estimated working hours of the task and the total number of manpower required. For example, if the initial prediction is 4,000 people, based on the project schedule, it is initially recommended to deploy 20 labelers.

[0098] S4.2: While the first deep learning model is performing operations, its hidden state vector is captured in real time and input into a dynamic splitting module; the dynamic splitting module includes a complexity analysis unit, which is used to calculate the data complexity coefficient and annotation difficulty fluctuation value of each data block in the coarse-grained task sample set.

[0099] In this embodiment, while the network based on the gated recurrent unit performs forward computation, its hidden state vector is captured in real time and input into a dynamic splitting module. The hidden state vector contains the network's memory information of the current task sequence. The dynamic splitting module includes a complexity analysis unit. The complexity analysis unit uses lightweight rules to quickly calculate each logical data block in the coarse-grained task sample set based on the information contained in the hidden state vector and combined with image metadata, such as the size, average brightness, and color distribution of each image. For example, 100,000 images are divided into 100 data blocks according to the collection road segment. The data complexity coefficient and annotation difficulty fluctuation value of approximately 1,000 images in each data block are used. The data complexity coefficient reflects the average complexity of the images in the data block, and the annotation difficulty fluctuation value reflects the standard deviation of the image difficulty in the data block from the average difficulty.

[0100] S4.3: When the complexity analysis unit identifies that the data complexity coefficient or annotation difficulty fluctuation value of any data block exceeds the preset threshold, it triggers the sample set splitting operation to extract the corresponding data block from the coarse-grained task sample set to form a fine-grained task sample set.

[0101] In this embodiment, the complexity analysis unit continuously monitors these coefficients and fluctuation values. When the data complexity coefficient or annotation difficulty fluctuation value of any data block exceeds a preset threshold—for example, if any data block contains a complex intersection and a large number of pedestrians, and its data complexity coefficient exceeds the threshold of 1.8—a sample set splitting operation is immediately triggered. The system extracts this data block from the coarse-grained task sample set as a high-complexity data block and separates it to form an independent fine-grained task sample set. In this example, approximately 20 high-complexity data blocks may be identified, and approximately 20,000 images are classified into the fine-grained task sample set.

[0102] S4.4: Input the fine-grained task sample set and its corresponding micro-indicator subset into the second deep learning model; the second deep learning model is a multilayer perceptron network, whose input layer is connected to the fully connected layer of the micro-indicator subset, the hidden layer uses the ReLU activation function, and the output layer outputs the fine-grained workload prediction value through the Sigmoid function, including the average time per sample, the proportion of data at different difficulty levels, and the time fluctuation range;

[0103] In this embodiment, a fine-grained task sample set and its corresponding micro-indicator subset are input into a second deep learning model. The fine-grained task sample set consists of 20,000 highly complex images, and the micro-indicator subset includes single-sample base processing time, data complexity multiplier, and accuracy correction coefficient. The second deep learning model is a multilayer perceptron network with a relatively simple structure, suitable for processing static feature inputs and performing regression prediction. The input layer of the multilayer perceptron network is connected to the fully connected layer of the micro-indicator subset. The hidden layers use the ReLU activation function to introduce nonlinearity, and the output layer uses the Sigmoid function to limit the output within a reasonable range, outputting fine-grained workload prediction values, including:

[0104] Average time per sample: The average annotation time for these 20,000 highly complex images, for example, 450 seconds per image, is much higher than the overall average estimate;

[0105] Data proportions at different difficulty levels: Predict the proportions of extremely high, high, and medium difficulty images among these 20,000 highly complex images;

[0106] Time fluctuation range: the standard deviation or confidence interval of the time to label a single image, for example, 350 seconds to 550 seconds.

[0107] S4.5: The output of the second deep learning model is fed back to the gated loop unit of the first deep learning model to correct the coarse-grained workload prediction value and obtain the final dynamically predicted total task time and time distribution.

[0108] In this embodiment, the output of the second deep learning model is fed back to the gated recurrent unit of the first deep learning model in real time. The specific process includes:

[0109] (1) Compare the average time per sample in the fine-grained workload prediction with the average time per sample estimated based on the overall average in the coarse-grained prediction. The average time per sample is 450 seconds, and the average time per sample calculated based on the total working hours and the total number of maps is about 144 seconds.

[0110] (2) Combined with the fluctuation range of time consumption, it is converted into an efficiency correction factor. For example, 450 / 144≈3.125, which means that the efficiency of high-complexity data block is only about 1 / 3.125 of the average level.

[0111] (3) Perform a dot product operation between the efficiency correction factor and the hidden state vector of the first deep learning model at the current time step. Adjust the feature weights related to task time prediction in the hidden state vector through this dot product operation to correct the prediction logic of the gated loop unit. This correction is based on the fine-grained prediction results. It is clear that there is a positive bias in the efficiency estimation of high-complexity data blocks in the coarse-grained prediction stage. It is necessary to dynamically update the memory weights and prediction parameters of the gated loop unit for task time-related features through weight adjustment so that the first deep learning model can fully incorporate the low efficiency characteristics of high-complexity data blocks.

[0112] (4) After feedback correction, the first deep learning model re-evaluates the overall task. Now it takes into account the efficiency of 80,000 medium-complexity images and the actual situation of 20,000 low-efficiency high-complexity images, and outputs the final dynamic prediction of the total task time and time distribution. For example, it is corrected from 4,000 person-hours to 5,200 person-hours. The normal data block is expected to take 3,800 person-hours, and the high-complexity data block is expected to take 1,400 person-hours. For example, 1,400 person-hours means that a worker needs to work continuously for 1,400 hours to complete the task.

[0113] The output of the second deep learning model is fed back to the gated recurrent unit of the first deep learning model. The process of correcting the coarse-grained workload prediction includes:

[0114] The average time and time fluctuation range of a single sample in the fine-grained workload prediction value are converted into an efficiency correction factor; the efficiency correction factor is multiplied by the hidden state vector of the first deep learning model to adjust its memory and prediction of the total task time.

[0115] The automatic allocation of annotation personnel, tools, and time resources based on dynamic prediction results includes:

[0116] S5.1: Analyze the total task time and time distribution of the dynamic prediction results, and generate a resource requirement list. The resource requirement list clearly lists the skill level and number of personnel to be labeled, the type and computing power requirements of the labeling tools, and the time window for each stage.

[0117] In this embodiment, the required total person-days for annotation are 5200 person-days divided by 8 hours / day, resulting in 650 person-days. Considering parallel work, the total number of annotators needed is approximately 650 person-days / 20 days ≈ 33 annotators. Furthermore, based on task complexity, the resource requirements list clearly specifies the skill levels these annotators must possess: at least 20 with advanced image annotation skills for handling highly complex data blocks, and the remaining 13 with intermediate skills for handling ordinary data blocks.

[0118] Annotation tools: The type of annotation tool required and its computing power requirements. In this example, it is an image annotation tool, such as LabelImg, CVAT, or a customized platform. For high-precision and high-complexity target detection tasks, stronger computing resources are needed for real-time rendering and AI-assisted pre-annotation.

[0119] Time resources: Time windows for each stage. For example, the first week is for personnel training and data familiarization, the second to fourth weeks are for concentrated annotation, and the last two days are reserved for quality spot checks and rework.

[0120] S5.2: Match the resource requirement list with a real-time resource pool, which dynamically updates the status of currently available annotators, tool load, and time slot information;

[0121] S5.3: Optimal matching calculation is performed based on the Hungarian algorithm, allocating appropriate annotators, idle annotation tool instances, and continuous time windows to each annotation task sub-item, and generating a resource allocation scheme. The calculation process of the Hungarian algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0122] Furthermore, the process of optimal matching calculation based on the Hungarian algorithm includes: modeling the annotation task sub-items and resources as a bipartite graph; allocating suitable annotators, idle annotation tool instances, and continuous time windows to each data block by calculating the minimum cost or maximum benefit matching method. For example, high-complexity data blocks are preferentially assigned to highly skilled annotators with high historical efficiency, and they are allocated computing power instances equipped with high-performance GPUs and Faster R-CNN models to accelerate their annotation process; ordinary data blocks are assigned to intermediate annotators, using standard CPU computing power instances. Simultaneously, specific start and end times are planned for each block. In this embodiment, the annotation task sub-items consist of 100 data blocks, and the resources include 33 annotators and various tool instances.

[0123] When multiple labeled task sub-items compete for the same high-quality resource based on the Hungarian algorithm matching, a priority arbitration mechanism is set up, including: prioritizing the allocation of task sub-items with higher difficulty levels; if the difficulty levels are the same, prioritizing the allocation of task sub-items with more urgent deadlines; and automatically entering the next round of matching queue for task sub-items that fail to obtain the optimal resource due to conflict within the time window allowed by their time.

[0124] Furthermore, after the calculation is completed, the system generates a detailed resource allocation plan, including which image blocks each annotator is responsible for, what tools to use, and what time period to complete the task. The tool allocation information and annotation task data are sent to the annotator's client. After the annotator logs into their client, the system automatically loads the allocated tool environment for them. For example, it directly opens the annotation software interface with the pre-trained model configured and loads the list of image data assigned to them.

[0125] S5.4: Execute the resource allocation scheme and compare the allocation results with the subsequent actual annotation progress data. Use the comparison difference as feedback data to adjust the weight parameters of the first deep learning model and the second deep learning model.

[0126] In this embodiment, the system executes a resource allocation scheme and begins monitoring the actual annotation progress. Annotators complete annotations on the client, and the system records data such as actual time consumption and annotation quality. These actual annotation progress data are compared with the prediction results of S4 to generate comparison differences. For example, it is found that the actual time consumption for a certain type of complex scenario is 10% longer than predicted. These differences are used as feedback data and are used weekly for incremental learning or fine-tuning of the weight parameters of the first and second deep learning models. This allows the AI ​​prediction model to continuously learn from actual projects over time, becoming increasingly accurate in adapting to the organization's specific work patterns and data types, forming a self-optimizing closed-loop system.

[0127] When performing optimal matching calculations based on the Hungarian algorithm, if multiple labeling task sub-items compete for the same resource, a priority arbitration mechanism is set up, including: prioritizing the allocation of labeling task sub-items with high difficulty level, with priority order being high difficulty, medium difficulty, and low difficulty; if multiple labeling task sub-items have the same difficulty level, prioritizing the allocation of the labeling task sub-item with the shortest remaining available labeling time, where the remaining available labeling time is the difference between the task deadline and the current system time; for task sub-items that do not obtain the corresponding resource due to conflict, they automatically enter the next round of matching queue within the allowed time window.

[0128] The allocation of the annotation tool instances is specifically targeted at the target detection task in image annotation: a GPU computing power instance equipped with a large pre-trained model is allocated to the first annotation task sub-item; a standard CPU computing power instance is allocated to the second annotation task sub-item; the allocation information and annotation task data are sent to the annotation personnel's client together, and the allocated tool environment is automatically loaded after the annotation personnel log in. Among them, the first annotation task sub-item is the annotation task sub-item that requires high-precision annotation, and the second annotation task sub-item is the regular annotation task sub-item.

[0129] This embodiment uses a specific urban street scene multi-object detection and annotation task as a central theme, detailing the complete process from data acquisition, feature extraction, difficulty assessment, multi-dimensional workload prediction to automated resource allocation and closed-loop optimization. Through the deep application of AI technology, this method achieves intelligent, precise, and efficient annotation project management, improving resource utilization and project controllability. The above description is merely one specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this invention should be included within the scope of protection of this invention.

[0130] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the present invention. All of these variations are within the protection scope of the present invention.

Claims

1. A method for dynamic prediction and resource allocation of workload for labeled tasks, characterized in that, include: Obtain annotation task data; The labeled task data contains multiple key task features; Based on the extracted key features of the task, the core attributes and difficulty level of the labeled task are determined. Based on the established core attributes and difficulty levels, the annotation task is divided into coarse-grained workload dimension and fine-grained workload dimension; the core attributes include text annotation, image annotation, speech annotation, single-label annotation, and multi-label annotation; Based on coarse-grained and fine-grained workload dimensions, an AI model is used to dynamically predict the workload of the annotation task, and the predicted total task time and time distribution are output. Based on the dynamic prediction results, annotation personnel, tools, and time resources are automatically allocated; The process of dividing the coarse-grained workload dimension and the fine-grained workload dimension includes: based on the core attributes and difficulty level output by the fully connected layer classifier, querying and determining the core dimension indicator set for workload prediction from a pre-configured dimension indicator mapping table; the core dimension indicator set includes a macro-indicator subset for coarse-grained prediction and a micro-indicator subset for fine-grained prediction; constructing the coarse-grained workload dimension based on the macro-indicator subset, and constructing the fine-grained workload dimension based on the micro-indicator subset; The method of dynamically predicting the workload of labeling tasks using AI models and outputting the predicted total task time and time distribution includes: Based on the coarse-grained workload dimension, the preprocessed labeled task data is used as a coarse-grained task sample set and input into the first deep learning model. The first deep learning model is a network based on gated recurrent units. Its input layer receives the macro-indicator subset, and its output layer outputs the coarse-grained workload prediction value, including the total estimated task hours and the total manpower requirement. While the first deep learning model is performing calculations, its hidden state vector is captured in real time and input into a dynamic splitting module; the dynamic splitting module includes a complexity analysis unit, which is used to calculate the data complexity coefficient and annotation difficulty fluctuation value of each data block in the coarse-grained task sample set. When the complexity analysis unit identifies that the data complexity coefficient or annotation difficulty fluctuation value of any data block exceeds a preset threshold, it triggers a sample set splitting operation to extract the corresponding data block from the coarse-grained task sample set to form a fine-grained task sample set. The fine-grained task sample set and its corresponding micro-indicator subset are input into the second deep learning model; the second deep learning model is a multilayer perceptron network, whose input layer is connected to the fully connected layer of the micro-indicator subset, the hidden layer uses the ReLU activation function, and the output layer outputs the fine-grained workload prediction value through the Sigmoid function, including the average time per sample, the proportion of data at different difficulty levels, and the time fluctuation range; The output of the second deep learning model is fed back to the gated recurrent unit of the first deep learning model to correct the coarse-grained workload prediction value, so as to obtain the final dynamically predicted total task time and time distribution.

2. The method for dynamic prediction and resource allocation of annotation task workload as described in claim 1, characterized in that, The process of determining the core attributes and difficulty level of the labeled task based on the extracted key features includes: The labeled task data is input into the feature extraction model, which outputs a key feature vector for the task. The key feature vector includes the data size quantization value, data type encoding, label type encoding, single sample information density value, and historical average labeling efficiency. The feature extraction model consists of three layers of convolutional neural network and two layers of long short-term memory network connected in sequence. The convolutional neural network is used to extract local numerical features in the data size and single sample information density, while the long short-term memory network is used to learn the label type and the sequence dependency relationship in the historical labeled data. The key feature vectors of the task are input into a fully connected layer classifier, which outputs core attributes and difficulty level. The core attributes include text annotation, image annotation, speech annotation, single-label annotation, and multi-label annotation, which are jointly determined by the data type encoding and the annotation type encoding. The difficulty level includes low, medium, and high difficulty levels, which are calculated by the fully connected layer classifier based on the data scale quantization value, the single sample information density value, and the historical average annotation efficiency.

3. The method for dynamic prediction and resource allocation of annotation task workload as described in claim 2, characterized in that, In the feature extraction model, the specific configuration of the three-layer convolutional neural network is as follows: the first convolutional layer uses 64 filters of size 3×1 with a stride of 1, and the output feature map is downsampled by a max pooling layer; the second convolutional layer uses 128 filters of size 3×1; the third convolutional layer uses 256 filters of size 3×1; the two subsequent long short-term memory (LSTM) networks have 128 and 64 hidden units respectively, and the final hidden state of the last LSM network serves as the sequence information summary of the key feature vector of the task.

4. The method for dynamic prediction and resource allocation of annotation task workload as described in claim 3, characterized in that, In the feature extraction model, the specific rules for calculating the difficulty level of the fully connected layer classifier are as follows: The initial difficulty score is obtained by multiplying the quantified value of the data size by the information density value of a single sample. If there are historical annotation tasks that match the current task to a preset matching threshold and the historical average annotation efficiency is lower than the system benchmark efficiency, then the initial difficulty score is weighted and increased according to the degree to which it is lower than the system benchmark efficiency to obtain the increased difficulty score. Based on the range of the increased difficulty score, it is mapped to low, medium, and high difficulty levels.

5. The method for dynamic prediction and resource allocation of annotation task workload as described in claim 1, characterized in that, When constructing a coarse-grained workload dimension based on the macro-indicator subset, the coarse-grained workload dimension includes the overall data volume of the task, the unit data volume weight coefficient based on the core attributes, and the baseline efficiency adjustment coefficient calculated based on the difficulty level and historical average annotation efficiency. When constructing a fine-grained workload dimension based on the subset of micro-indicators, the fine-grained workload dimension includes the basic time consumption per sample calculated based on the single-sample information density value and the annotation type encoding, the data complexity multiplier determined based on the difficulty level, and the accuracy correction coefficient derived from the annotation accuracy requirements.

6. The method for dynamic prediction and resource allocation of annotation task workload as described in claim 5, characterized in that, The output of the second deep learning model is fed back to the gated recurrent unit of the first deep learning model. The process of correcting the coarse-grained workload prediction includes: The average time and time fluctuation range of a single sample in the fine-grained workload prediction value are converted into an efficiency correction factor; the efficiency correction factor is multiplied by the hidden state vector of the first deep learning model to adjust its memory and prediction of the total task time.

7. The method for dynamic prediction and resource allocation of annotation task workload as described in claim 1, characterized in that, The automatic allocation of annotation personnel, tools, and time resources based on dynamic prediction results includes: The total task time and time distribution of the dynamic prediction results are analyzed to generate a resource requirement list. The resource requirement list is matched with the real-time resource pool, which dynamically updates the current status of the annotators, the tool load, and the time slot information. The optimal matching calculation is performed based on the Hungarian algorithm, and annotation personnel, idle annotation tool instances and continuous time windows are allocated to each annotation task sub-item, and a resource allocation plan is generated. The resource allocation scheme is executed, and the allocation results are compared with the actual annotation progress data. The difference between the comparison and the data is used as feedback data to adjust the weight parameters of the first deep learning model and the second deep learning model.

8. The method for dynamic prediction and resource allocation of annotation task workload as described in claim 7, characterized in that, When performing optimal matching calculations based on the Hungarian algorithm, if multiple labeling task sub-items compete for the same resource, a priority arbitration mechanism is set up, including: prioritizing the allocation of labeling task sub-items with high difficulty level, with priority order being high difficulty, medium difficulty, and low difficulty; if multiple labeling task sub-items have the same difficulty level, prioritizing the allocation of the labeling task sub-item with the shortest remaining available labeling time, where the remaining available labeling time is the difference between the task deadline and the current system time; for task sub-items that do not obtain the corresponding resource due to conflict, they automatically enter the next round of matching queue within the allowed time window.

9. The method for dynamic prediction and resource allocation of annotation task workload as described in claim 8, characterized in that, The allocation of the annotation tool instances is based on the target detection task in image annotation: a GPU computing power instance equipped with a pre-trained model is allocated to the first annotation task sub-item; a standard CPU computing power instance is allocated to the second annotation task sub-item; the allocation information and annotation task data are sent to the client together, and the allocated tool environment is automatically loaded.

Images