39 results about "Crowdsourced testing" patented technology

The invention provides a recommendation method for improving Top-k crowdsourcing test platform tasks, and relates to the field of software test. The method comprises the following steps of: constructing an index model of testers through indexes for carrying out portrait modeling on the testers; constructing a judging matrix to carry out consistency inspection; and generating a recommendation listthrough calculating a relative weight of each layer and personnel evaluation data structure matching and scoring. According to the method, the platform availability and long-term development can be greatly enhanced; crowdsourcing testers can be matched to proper test tasks conducive to level improvement, so that the enthusiasm for learning test businesses can be improved; and high-quality test results can be obtained. Meanwhile, the demand for processing abnormal and complex scenes by mobile application crowdsourcing test platforms is solved.

The invention provides a crowdsourcing test method. The crowdsourcing test method comprises the following steps: a test client side is used for sending characteristic information of the test client side to a test server; the test server is used for selecting a to-be-tested client side used for executing to-be-tested service according to the received characteristic information of at least one test client side; the test server is used for sending a to-be-tested service list to the to-be-tested client side; the to-be-tested client side is used for testing the to-be-tested service in the to-be-tested service list to obtain a test result of the to-be-tested service; and finally, the to-be-rested client side is used for sending the test result to the test server. The crowdsourcing test method used for testing network application service is used for remarkably improving the test result and effectively reducing the test cost.

The invention discloses a text description generating method, system and device of a crowdsourcing test report. The method comprises the steps that screenshots and text descriptions in the test reportare acquired, key word sequences used for embodying test information in the text descriptions are determined, and hierarchical clustering is conducted on the screenshots according to a preset hierarchical clustering standard to obtain a plurality of sets of screenshot class clusters; the key word sequences which correspond to the test report and to which each screenshot in each set of screen capture class cluster belongs are input to a probability language model, and a class cluster text description generated by the probability language model is obtained; according to a preset classificationstrategy, the screenshot class cluster to which each to-be-diagnosed screenshot belongs in the to-be-diagnosed test report is determined, and a class cluster text description corresponding to the screenshot class cluster to which the to-be-diagnosed screenshot belongs is obtained. The number of the screenshots is large, the obtained text descriptions are rich, a developer can understand the test report, and the efficiency of diagnosing and understanding the test report is improved for the developer; the screenshots are objective, so that the diagnosis and understanding of the developer are noteasy to mislead.

The invention discloses a mobile application crowdsourcing test report automatic evaluation method and a computer storage medium, and the method comprises the following steps: 1), inputting a test report set and the historical credibility of a worker, removing an invalid test report, and carrying out the word segmentation and stop word removal of the remaining test report set; 2) clustering the test reports according to the discovered defects to form a plurality of types of defect test reports, and selecting the grade of the maximum proportion as the defect grade of the type of defect test reports; 3) constructing a plurality of normativity indexes and corresponding step type metric functions, evaluating the test report, and converting the evaluation into a normativity score of the test report; 4) according to the defect grade and the normativity score, obtaining the final score of the test report. The defect that an existing crowdsourcing test report quality evaluation method lacks test report content quality evaluation is overcome, and the overall performance of a crowdsourcing test platform is improved.

The disclosure provides a method and an electronic device for recommending crowdsourced tester and crowdsourced testing. The method comprises: collecting a requirement description of a crowd testing task at a time point in a process of crowd software testing and historical crowd testing reports of each tester to be recommended; obtaining a process context and a resource context of each tester to be recommended; inputting the extracted features into a learning to rank model to obtain an initial ranking list of the recommended testers, and re-ranking the initial ranking list based on diversity contributions of expertise and device to obtain a final ranking list. The disclosure can more accurately recommend testers to take accuracy and diversity of the recommended testers into consideration, so that the testers can be dynamically planned during the crowd testing to improve the bug detection rate, shorten the completion cycle of the crowd testing task.

The invention discloses a voice recognition system crowdsourcing test case generation method based on test requirements. The method is characterized in that a provided voice recognition system is tested based on a crowdsourcing technology, the test requirements submitted by a task requester are recognized and extracted, a voice recognition keyword list in a preset keyword library is used for filtering out texts related to test case content from recognized text information fragments, and the texts serve as candidate test requirements. Three parts are included: a voice recognition test parametercollection module, a text extraction module and a test case generation module. Input is a text test demand, feature extraction is performed based on a user demand, and a test template is generated. The method has the following beneficial effect that the multi-dimensional and diversified requirements of ASR testing are met. The test participation threshold of crowdsourcing workers is reduced, testtasks are not limited by places and identities, and the crowdsourcing workers can sign up to participate in the test as long as the crowdsourcing workers have test tools and test capabilities, so that the test loss is reduced, and the product research and development period is shortened.

The invention discloses a crowdsourcing system-oriented simulation test case generation method and device, and the method comprises the steps: firstly inputting a sample obtained by a crowdsourcing platform into a VAE encoder, and learning a mean value and a variance of worker capability and task difficulty labeled for a real sample, namely obtaining the normal distribution of the corresponding worker capability and task difficulty; meanwhile, generating Gaussian mixture distribution of the capabilities and task difficulties of all workers participating in labeling; secondly, obtaining hidden variables of the VAE by using a reconstruction technique, inputting the hidden variables into a decoder of the VAE, and training the decoder by using gradient descent; and finally, inputting the worker ability and task difficulty randomly generated by Gaussian mixture distribution into a trained decoder, and generating a large number of crowdsourcing test cases which are independent and identically distributed with a real sample. According to the method, the neural network is combined to solve the problem that the crowdsourcing test cases are few and the coverage is narrow due to the cost, so that the result convergence method is difficult to comprehensively evaluate, and the crowdsourcing result convergence method can be comprehensively and effectively evaluated.

The invention relates to a device and method for processing running contexts of an application in a crowdsourcing test. The device comprises a crash monitoring module, an interaction module, a compression module and an upload module, wherein the crash monitoring module is configured to detect the crash of the application; the interaction module is configured to detect the selection of a tester to the active feedback by receiving the input of the tester; the compression module is configured to compress the behavior information of the recorded running contexts in response to the crash of the application which is detected or the mode that the tester selects active feedback so as to obtain the running contexts to be submitted; the upload module is configured to upload the running contexts to be submitted to a test platform; the step of compressing the behavior information in the recorded running contexts comprises the process of deleting the description of at least one event from the running contexts and simplifying at least one of the running contexts based on the repeated content in the running contexts; and the recorded running contexts contain activities related to the application and the behavior information of the event that occurs in the activities.

The present disclosure provides an information processing apparatus, an information processing method, and an information processing device for crowdsourcing tests. The information processing apparatus comprises: an extraction unit, for extracting test scenarios from a test record stored in a crowdsourcing test database, wherein the test scenario comprises an operation sequence of crowdsourcing testers; a clustering unit, for, according to the operation sequence of the crowdsourcing testers, clustering the extracted test scenarios to obtain a plurality of test scenario categories; and a determination unit, for determining the importance of each test scenario category to obtain test cases based on the importance. According to the information processing apparatus, the information processing method, and the information processing device provided by the present disclosure, at least important test cases can be effectively selected, so that the efficiency of crowdsourcing tests can be improved.

An Android crowdsourcing test feedback method based on astep prompt is characterized in that a user test behavior database is established through recording a test process by a driver of a crowdsourcing test platform Kikbug; in the test process, the test step information of the current test personnel is collected and compared with the recorded and triggered abnormal operation information in the database, so that the problem that a crowdsourcing test report is not professional enough is solved, and the test personnel are guided to complete a more efficient crowdsourcing test. The method comprises the steps that (1) establishing an abnormal test behavior database, completing all to-be-tested tasks through a task provider, and initializing the database; (2) comparing the current tester with anoperation sequence in a database, and guiding the tester to trigger an abnormity; (3) storing new abnormities appearing in the test process; (4) when the reproduction frequency of a certain abnormityexceeds a threshold value, defining the abnormity as a confirmed abnormity, and not guiding a tester to verify the abnormity any more; and (5) feeding back the abnormity to the task provider.

The invention discloses a Chinese crowdsourcing test report clustering method based on semantic similarity, and the method comprises the steps: inputting a Chinese test report set, removing invalid test reports, and carrying out the sentence segmentation of effective test reports, and obtaining a test report sentence pair data set; constructing a test report sentence pair training set to train the semantic similarity model to obtain a semantic similarity calculation model; inputting the test report sentence pair data set into a semantic similarity calculation model for semantic similarity calculation to obtain a test report similarity matrix; setting an expected bug number of the test item, and performing spectral clustering according to the test report similarity matrix to obtain a test report class cluster; and decomposing the test report similarity matrix according to the test report class clusters to obtain the test report similarity matrix of each class cluster, and calculating a test report with an accumulated similarity score Top-5 in each class cluster as a final output result. The test report review automation process of the crowdsourcing test platform is improved, and the test report review efficiency is effectively improved.

The invention provides a cooperative crowdsourcing test oriented quality control method, and realizes a cooperative crowdsourcing test oriented quality control system on the basis of the cooperative crowdsourcing test oriented quality control method. The method is characterized in that 1, each operation of the crowdsourcing workers is evaluated and fed back in real time, so the quality awareness of the crowdsourcing workers is improved, and the disastrous behaviors of the crowdsourcing workers are recorded; 2, the quality grade of the report is determined through automatic scoring of the report, and a favorable reference is provided for managers to identify effective reports; and 3, a report auditing function is provided, so that a manager can conveniently check all information of the report and quickly and accurately auditing the report. The invention has the beneficial effects that the quality consciousness of crowdsourcing workers is improved through various designed quality controlmethods, the ratio of effective Bug reports in test tasks is improved, and managers are helped to identify the Bug reports and the quality of the crowdsourcing workers more quickly and accurately.

The embodiment of the invention provides a crowdsourcing test report recommendation method and device, and electronic equipment. The method comprises the following steps: obtaining a first defect report and a plurality of historical defect reports, determining a first feature vector of the first defect report and a second feature vector of each historical defect report, determining the similarity between the first feature vector and each second feature vector, and sorting the similarities; the historical defect reports are recommended on the basis of the sorting result, whether the defect reports similar to the first defect report exist in the historical defect reports or not can be determined, and therefore the defect reports with repeated content are prevented from occurring in the process that a user edits the first defect report, and the accuracy of the crowdsourcing test task is improved.

The invention discloses a fairness-oriented crowdsourcing tester recommendation method and device. The method comprises the following steps: representing a demand description of a crowdsourcing task as a descriptive term vector; according to the descriptive term vector and the attribute information of the to-be-recommended personnel, predicting the defect detection probability of each to-be-recommended personnel in the crowdsourcing task; and setting a plurality of fairness targets including maximization of the defect discovery probability and minimization of the recommendation frequency difference, and performing multi-target optimization to obtain a crowdsourcing tester recommendation result. According to the method, the unfairness problem can be relieved, more fair recommendation is generated, and on the basis of generating the more fair recommendation, the defect detection efficiency is not influenced, and the defect detection efficiency is improved.

The invention discloses a crowdsourcing test report clustering method based on an AP affinity propagation algorithm. According to the method, firstly, keywords are extracted by utilizing a natural language processing technology, and identical or similar words are replaced by the same word by utilizing a synonym forest, so that the semantic deviation problem caused by different words is effectively solved. Then an asymmetric similarity strategy is adopted to calculate the similarity between test reports, and the strategy can effectively solve the problem that the similarity calculation is inaccurate under the condition that the length difference of the test reports is large. Finally, the test reports are clustered by using an AP affinity propagation clustering algorithm. Through the method, a crowdsourcing test report clustering effect can be improved, a developer only needs to select one test report from each cluster for examination, the examination time of the developer is greatly shortened, and the method has very important significance on software maintenance.

The invention discloses a crowdsourcing test task completion degree evaluation method and system. The method comprises the following steps: obtaining a test cost element data set according to test process conditions; performing correlation analysis on the test manpower cost data set and the test report cost data set; obtaining a defect data set of a test report result, and selecting a corresponding modeling equation from a pre-constructed segmented reliability model framework based on the correlation degree of a test manpower cost data set and a test report cost data set according to a correlation analysis result, estimating parameters of the modeling equation by utilizing a least square and maximum likelihood estimation method based on the defect data set to obtain the total number of potential defects of the software; according to the actual detection defect number of the defect data set and the total number of the potential defects of the software, the detection defect coverage rate is obtained through calculation, and the task completion degree is evaluated. The method has the advantages that the quality of test software is guaranteed, waste of a large amount of test time and test resources can be avoided, and the overall performance of a crowdsourcing test platform is improved.

The invention discloses a crowdsourcing test task allocation method based on student ability matching, which depicts student ability based on multiple dimensions and quantitatively evaluates the student ability level of each dimension through a fuzzy analytic hierarchy process. A test task attribute is taken as a demand guide to assign a weight of each dimension capability level, and a comprehensive capability matching value is calculated to serve as a matching basis for distributing crowdsourcing test tasks, so that a test platform can formulate an accurate and reasonable crowdsourcing test distribution strategy, the efficiency of the crowdsourcing test tasks is effectively improved, the quality of a test report is guaranteed, and the purposes of generating industry and education deep cooperation, providing professional talents for the industry and improving talent training quality for colleges and universities are achieved.

A crowdsourcing test report similarity detection method based on natural language processing adopts a natural language processing technology to detect the similarity of complex test reports submitted by crowdsourcing workers, and has the function of performing Chinese word segmentation, stop word removal and other preprocessing on the crowdsourcing reports. Sentences represented by the preprocessed phrases are represented as word vectors by utilizing a Word2Vec technology, a cosine similarity measurement mode is selected to calculate the distance between the word vectors, training is performed by adopting a semantic model trained according to a large amount of previous crowd test report data, and then each word vector is taken as input of K-Means clustering analysis, and clustering analysis is carried out on each word vector, and similar reports are classified into the same class according to a set similarity threshold, so that the similarity between the crowdsourcing test reports can be accurately measured.

A crowdsourcing test report fusion method based on public and private information mining comprises the steps that (1) feature mapping is carried out, each mode comprises public information shared by all modes and private information of the modes, and the modes extracted from an original feature space are mapped into public vectors and private vectors of the same dimension; (2) public information shared by all the modes included in different modes and private information of each mode are obtained; 3) modal relationship mining is carried out, and the obtained d-dimensional public information Ci and private information Sim are sequentially input into the LSTM for processing; and 4) label prediction is carried out, in multi-label learning, each mode has a specific contribution to multi-label prediction, and the relationship between labels is obtained by using the information of the different modes stored in an LSTM memory unit. According to the method, the low-dimensional common representation of all modes can be learned, the distinguishing information of each mode is extracted through forced orthogonal constraint, meanwhile, the common representation serves as a new mode to be added into a specific mode sequence, and label correlation mining based on the extracted mode sequence is achieved.

A multi-path recall recommendation method for crowdsourcing software testing comprises the steps: S1, conducting data collection, and obtaining a data set for multi-path recall, wherein the data set comprises user information, user social information, project information, project text description and user historical project completion conditions; S2, preprocessing the data, and calculating an input number for a multi-path recall model; S3, performing project multi-path recall, finding similar users of a target user by calculating user similarity, and finding historical completion projects of the target user and the similar users through a user-project interaction matrix; S4, performing user multi-path recall, finding similar items of a target item by calculating the item similarity and the item text embedding similarity, finding users completing the target item and the similar items through a user-item interaction matrix, and finding similar users of the users by calculating the user similarity and a user relation matrix. Thus, multi-path recall of crowdsourcing test users is completed.