Public test task allocation technology based on recommendation algorithm

Abstract

The invention relates to the field of artificial intelligence and industrial control safety, in particular to a public test task allocation technology based on a recommendation algorithm. The technology aims to efficiently distribute the crowd test tasks to suitable testers or test teams by combining user historical records, user features and task features, and an efficient, accurate and fair crowd test task distribution system is constructed by combining the advantages of various recommendation algorithms. Firstly, collected task data and user data are converted into vector representation by using an NLP technology, then a recommendation algorithm based on attention, an MLP and an LSTM structure is constructed in combination with task features, user features and historical records for crowd test task allocation, and the crowd test task allocation efficiency is improved by adjusting hyper-parameters of a model and a structure training model. And finally, generating a personalized task allocation list for each user, collecting and recording feedback and results of the users on the allocated tasks, and updating historical records and features of the users for decision making during next task allocation.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and industrial control security, and specifically to a crowd-testing task allocation technology based on a recommendation algorithm. Background Art

[0002] As the scale of software development continues to expand, the importance of software quality assurance has become increasingly prominent. Crowdsourcing Testing, as an emerging software testing model, uses testers from all over the world to conduct testing, which can not only more efficiently discover defects in the software, but also verify the stability of the software in different usage scenarios [1]. However, with the increase in the number of crowdsourcing tasks and the diversity of testers' backgrounds, how to efficiently assign crowdsourcing tasks to the most suitable testers has become a challenge that needs to be solved [2]. Traditional crowd testing task allocation methods usually rely on the tester's autonomous selection or simple random allocation. Although this method is easy to operate, it has many problems. First, manual task allocation is subjective and easily affected by personal preferences and experience, resulting in inaccurate task allocation. Second, the task allocation process requires a lot of time and manpower costs, which limits the efficiency and scale of task allocation. For example, testers may not be able to effectively find defects in the software because they are not familiar with certain fields, or some tasks may not be accepted due to improper allocation [3]. With the increasing number of participants in crowd testing platforms, how to efficiently and accurately allocate test tasks to appropriate testers or test teams has become an important challenge. In order to address these issues, in recent years, recommendation algorithms have begun to be introduced into crowd-testing task allocation to improve allocation efficiency and accuracy. Recommendation algorithms analyze users' historical behaviors, interests, and preferences, and combine other users' behavioral data to predict the content that users may be interested in and make personalized recommendations [4]. In crowd-testing task allocation, recommendation algorithms can recommend the most suitable testers for each test task based on factors such as the testers' historical test records, skill levels, areas of interest, and experience, thereby improving test efficiency and enhancing test results [5]. In addition, recommendation algorithms can also dynamically adjust allocation strategies based on real-time feedback to optimize the utilization efficiency of test resources.

[0003] However, the application of existing recommendation algorithms in crowd-testing task allocation still faces some challenges. First, the diversity and complexity of test tasks require recommendation algorithms to have high accuracy and robustness to ensure that tasks can be assigned to testers with corresponding skills. Second, the dynamic changes of testers (such as new testers or the improvement of the skills of existing testers) require recommendation algorithms to have good adaptability and scalability. Finally, to prevent bias in task allocation, recommendation algorithms must also consider fairness and transparency to ensure that each tester has a reasonable opportunity to be assigned tasks[6]. Based on the above background, the present invention proposes a crowd-testing task allocation technology based on recommendation algorithm[7], so as to more efficiently allocate crowd-testing tasks to suitable testers or test teams. This technology combines the advantages of multiple recommendation algorithms to build an efficient, accurate and fair crowd-testing task allocation system[8]. The system can intelligently allocate test tasks according to the skill level of testers, historical test records and specific requirements of tasks, optimize the allocation efficiency of test resources, and maximize the participation and enthusiasm of testers while meeting project requirements[9]. [1] Wang Qiang, Zhi Cijia, Xu Jia. Software crowd testing task allocation mechanism based on reputation system[J]. Journal of Nanjing University of Science and Technology, 2022, Vol. 46(5): 561-570. [2] Cui Qiang, Wang Junjie, Xie Miao, et al. Research on Worker Selection Method in Crowd Testing[J]. Journal of Software, 2018, Vol. 29(12): 3648-3664. [3] Deng Pingping. User portrait analysis and research based on crowd testing[J]. China Informatization, 2021, (3): 40-41. [4] Zhang Xiaofang, Feng Yang, Liu Di, et al. Research progress of crowdsourcing software testing technology[J]. Journal of Software, 2018, Vol. 29(1): 69-88. [5] Wang Tingyong, Huang Song. GUI software test case generation based on crowd testing operation records[J]. Computer Age, 2022, (3): 27-31. [6]Tsai WT, Zhang L, Hu S, et al. Crowdtesting Practices and Models: An Empirical Approach[J]. Information and Software Technology, 2023,154:107103. [7]Alyahya S.Collaborative Crowdsourced Software Testing[J].Electronics,2022,11(20):3340. [8]Wang J, Yang Y, Wang S, et al. Context-and fairness-aware in-processcrowdworker recommendation[J]. ACM Transactions on Software Engineering and Methodology (TOSEM), 2022, 31(3):1-31. [9]Yang P, Chang C, Tang Y. Crowdsourced Testing Task Assignment based Knowledge Graphs[C] / / 2022 IEEE 22nd International Conference on SoftwareQuality,Reliability and Security(QRS).IEEE,2022:663-671. Summary of the invention

[0004] The purpose of the present invention is to provide a crowd-testing task allocation technology based on a recommendation algorithm, which aims to use the TiLSTM model to efficiently allocate crowd-testing tasks by combining user history records, user characteristics and task characteristics. Specifically, the present invention proposes a crowd-testing task allocation technology based on the TiLSTM model, which converts user and task information into multi-dimensional vectors through an embedding layer, and optimizes the LSTM mechanism to capture the association between user behavior and task requirements. The technology generates task scores through hidden states to achieve personalized recommendations, aiming to improve crowd-testing efficiency, optimize user experience, and enhance the system's adaptability to new data to ensure the accuracy of task allocation.

[0005] The objective of the present invention is achieved through the following technical solutions: A crowd testing task allocation technology based on a recommendation algorithm, the technology comprising: Step 1: In the data preparation phase, collect task data and user data, and use NLP technology to convert the data into vector representation; Step 2: In the model building stage, combining task features, user features, and historical records, a recommendation algorithm based on attention, MLP, and LSTM structures is constructed for crowd-testing task allocation; Step 3: In the model training and evaluation phase, the model is trained using the data from the training set, and the model's hyperparameters and structure, such as the number of network layers, number of hidden units, and learning rate, can be adjusted based on the performance on the validation set. Step 4: In the task assignment phase, use the above model to generate a personalized task assignment list for each user; Step 5: In the task feedback phase, collect and record the user's feedback and results on the assigned tasks, and update the user's history records and features for decision-making during the next task assignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0007] Figure 1 The figure is a schematic diagram of a crowd testing task allocation technique based on a recommendation algorithm according to an embodiment of the present invention. Figure 2 The figure is a schematic diagram of the crowd testing task allocation process based on the recommendation algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0008] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, which does not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0009] Figure 1 A schematic diagram of a technical flow of crowd-testing task allocation based on a recommendation algorithm provided in an embodiment of the present invention, wherein the steps include: Step 1. In the data preparation stage, first collect and organize the user's history and task data, including the user's history data, including completed tasks, task difficulty, feedback, timestamps and other information. Extract user features, such as the user's skill level, participation history, preferences, etc. Collect the attributes and requirements of the task, including task type, complexity, expected completion time, etc. Secondly, perform data preprocessing. Clean the collected data to remove noise and incomplete data. Standardize or normalize user features and task features to facilitate subsequent model training. Perform feature embedding on the history data and task data, and convert them into vector form for model processing.

[0010] Step 2. In the model building stage, from bottom to top, they are: input layer, embedding layer, TiLSTM layer, MLP layer, attention layer and output layer. ① Input layer: receives the original data x of user features, task features and historical records. As the front end of the model, the input layer is responsible for acquiring and integrating all basic information that affects task assignment and recommendation decisions, and providing the necessary context for subsequent data processing; ② Embedding layer: converts the input feature data into a low-dimensional dense vector e, thereby effectively capturing the semantic relationship between features and improving computational efficiency; ③ TiLSTM layer: the input is a combination of historical records, user and task features, and the embedded output of the current time step. Through the joint action of the forget gate, input gate, and output gate, TiLSTM can effectively process sequence data, capture time series features, and take into account the specific characteristics of users and tasks. This makes it perform well in application scenarios such as task assignment and recommendation systems. In TiLSTM, the input vector of the current time step is composed of the following parts: Historical record The hidden state of the previous time step carries the information of the previous time step; user and task features p: features about the current user and task, usually some auxiliary information, which may include the user's historical behavior, preferences, etc.; embedding output e: the embedding representation of the current input, usually derived from the embedding layer, used to represent the context information of the current input. Therefore, the input vector can be expressed as: The concatenation operation [·] here combines these vectors into a single input vector. In this model, we set the historical record to a forward LSTM. On the one hand, because the historical records of crowd-testing users are time series data, and the unidirectional LSTM can effectively capture past information and reflect the user's behavior before a specific time point, it is helpful to understand the user's habits and preferences. On the other hand, the behavior after the historical record usually does not affect the previous behavior, so only the past time series information needs to be paid attention to. This structure avoids the interference of irrelevant information. User and task features are often relatively static in time, and the impact of the information contained at each time point is relatively independent, so there is no need to follow a strict time order. The core of TiLSTM lies in its gate mechanism, which allows the model to process input data efficiently. In task assignment, the user's historical behavior may contain some outdated or no longer relevant information. The forget gate can dynamically filter and remove this outdated information to avoid information overload. When new tasks or new user behaviors appear, the input gate allows TiLSTM to selectively write this information so that the model can quickly adapt to new changes, such as user preferences or behavior patterns. The output gate in TiLSTM controls which information is passed to subsequent layers. Only information that has an important impact on future tasks is passed, allowing the model to better match and prioritize tasks. The following are the definitions and calculations of each gate: 1. Forget gate: f t =σ(Wf ·x t +b f ), where W f is the weight of the forget gate, b f is the bias. 2. Input gate: i t =σ(W i ·x t +b i ), where W i is the weight of the input gate, b i is the bias. 3. Candidate memory cells: Where W c is the weight of the candidate memory unit, b c is the bias. 4. Memory unit update: Current memory cell c t It is obtained by the weighted combination of the memory unit and the input gate of the previous time step and the candidate memory. 5. Output gate: o t =σ(W o ·x t +b o ). 6. Hidden state update: h t =o t ⊙tanh(c t ), hidden state h t It is the adjustment result of the current memory unit through the output gate. ④MLP layer: transform the output h of TiLSTM t Passed to the multi-layer perceptron (MLP), processed by the hidden layer o, and multiple outputs y are generated. The MLP layer can further abstract and combine the high-dimensional features of TiLSTM through nonlinear transformation. This enables the model to learn more complex feature representations, thereby improving the accuracy of predictions. The multi-layer structure of MLP enhances the learning and expression capabilities of the model, which is crucial for handling complex tasks and feature combinations. ⑤ Attention layer: The attention mechanism is used to pay attention to multiple outputs y m The final output is generated by weighting. In the crowd-testing task allocation, the attention layer dynamically weights multiple potential task outputs through the attention mechanism. The model can generate the optimal task recommendation based on the current user's historical behavior, skills, and preferences. The introduction of the attention layer can improve the interpretability of crowd-testing task allocation. Users can better understand why a task is recommended to them, which enhances user trust. The input of the attention layer Y = [y1, y2, ..., y m ], y represents the output vector, tanh is the activation function, and the output y formula is as follows: M = tanh (Y), α = softmax (w m M),y=Yα m , where α1,α2,...,α mis the attention weight. ⑥ Output layer: Outputs the predicted task score or matching degree. The output layer is responsible for generating the recommended task score or matching degree, which is used to show the most suitable task selection to the user.

[0011] Step 3: During the model training and evaluation phase, use appropriate optimization algorithms such as Adam to update parameters, evaluate model performance on the validation set, and use indicators such as accuracy, recall, F1-score, etc. to perform model tuning and adjust model hyperparameters based on the evaluation results.

[0012] Step 4: In the task assignment phase, the trained model is used to recommend new users and generate a task assignment list. Tasks are prioritized based on user characteristics and historical records, combined with the recommendation results output by the model, to ensure that important and appropriate tasks are assigned to appropriate participants first, maximizing participant satisfaction and task completion quality.

[0013] Step 5: In the task feedback phase, participants’ feedback on the assigned tasks is collected, including completion status, quality evaluation, and satisfaction. The feedback information is updated to the historical records to form a new data set. The attention layer further analyzes the feedback data so that the model can focus on the characteristics of high-quality tasks. The scoring mechanism of the output layer can adjust the weights based on the feedback so that the recommendation results are more in line with user needs and task goals. Combining feedback collection, historical record updates, and model optimization, this crowd-testing task allocation technology based on the recommendation algorithm can achieve continuous improvement. The dynamic adjustment of the attention layer and the output layer ensures the accuracy and personalization of the recommendations, better meeting the expectations and needs of users. Through this closed-loop feedback mechanism, the entire system continues to evolve in practice, promoting the success and participation of crowd-testing projects.

[0014] As described above, the crowd-testing task allocation technology based on recommendation algorithm proposed in the present invention can effectively combine historical records, user characteristics and task characteristics, and combined with the historical record processing capabilities of unidirectional LSTM, MLP and attention mechanism, the model can deeply mine the historical performance information of participants, realize efficient task allocation and dynamic optimization, and provide more accurate recommendations for task allocation.

Claims

1. A crowd testing task allocation method based on recommendation algorithm, characterized in that: The following steps are involved: Step 1: Data preparation stage: collect task data and user data, and use natural language processing (NLP) technology to convert the data into vector representation; Step 2: In the model building phase, we combine task features, user features, and historical records to build a recommendation algorithm that includes an attention mechanism, MLP, and LSTM structures for crowd-testing task allocation. Step 3: Model training and evaluation phase: Use the training set data to train the model, and adjust the model's hyperparameters and structure based on the performance of the validation set; Step 4: Task assignment phase: Use the trained model to generate a personalized task assignment list for each user. Step 5: Task feedback phase: collect user feedback on assigned tasks and update user history records and features to optimize subsequent task assignment decisions.

2. The method for allocating crowd testing tasks according to claim 1, characterized in that: The model construction includes using an embedding layer to convert user and task information into a multi-dimensional vector representation, and optimizing the LSTM mechanism through a TiLSTM layer to effectively capture the association between user behavior and task requirements.

3. The crowd-testing task allocation method according to claim 1, characterized in that: In the task assignment step, the task assignment list generated by the model is matched based on the user's historical test records, skill level, interest areas, and specific requirements of the task.

4. The crowd-testing task allocation method according to claim 1, characterized in that: The task feedback step includes collecting user feedback information on task completion, quality evaluation and satisfaction, and updating this information into the user's historical record.

5. The crowd-testing task allocation method according to claim 1, characterized in that: In the model training and evaluation steps, the Adam optimization algorithm is used to update parameters, and the model performance is evaluated by indicators such as accuracy, recall rate and F1-score.

6. The crowd-testing task allocation method according to claim 1, characterized in that: The TiLSTM layer in the model building phase is able to process sequence data, capture time series features, and consider specific attributes of users and tasks to optimize task allocation strategies.

7. The crowd-testing task allocation method according to claim 1, characterized in that: The attention mechanism is used to dynamically weight multiple potential task outputs to generate optimal task recommendations. A crowd testing task allocation system based on the method according to any one of claims 1 to 7, comprising: Data collection module, used to collect user and task data; Model training module, used to build and train recommendation algorithm models; The task assignment module is used to generate a task assignment list for the user based on the trained model; Feedback processing module, used to collect user feedback and update historical records.

Images