39 results about "Crowdsourced testing" patented technology

The invention provides a recommendation method for improving Top-k crowdsourcing test platform tasks, and relates to the field of software test. The method comprises the following steps of: constructing an index model of testers through indexes for carrying out portrait modeling on the testers; constructing a judging matrix to carry out consistency inspection; and generating a recommendation listthrough calculating a relative weight of each layer and personnel evaluation data structure matching and scoring. According to the method, the platform availability and long-term development can be greatly enhanced; crowdsourcing testers can be matched to proper test tasks conducive to level improvement, so that the enthusiasm for learning test businesses can be improved; and high-quality test results can be obtained. Meanwhile, the demand for processing abnormal and complex scenes by mobile application crowdsourcing test platforms is solved.

The invention provides a crowdsourcing test method. The crowdsourcing test method comprises the following steps: a test client side is used for sending characteristic information of the test client side to a test server; the test server is used for selecting a to-be-tested client side used for executing to-be-tested service according to the received characteristic information of at least one test client side; the test server is used for sending a to-be-tested service list to the to-be-tested client side; the to-be-tested client side is used for testing the to-be-tested service in the to-be-tested service list to obtain a test result of the to-be-tested service; and finally, the to-be-rested client side is used for sending the test result to the test server. The crowdsourcing test method used for testing network application service is used for remarkably improving the test result and effectively reducing the test cost.

The invention discloses a method and device for processing crowdsourcing test data. The method comprises the following steps of: obtaining task description information of a to-be-tested application, wherein the task description information is used for indicating to carry out crowdsourcing test on the to-be-tested application; selecting a target tester set for testing the to-be-tested application from a to-be-selected tester set on the basis of a set of tester models and the task description information; and generating a crowdsourcing test strategy according to the target tester set and distributing the to-be-tested application to each tester in the target tester set according to the crowdsourcing test strategy. By adoption of the method and device, the efficiency of selecting testers in target tester sets can be improved, potential testers with good test resources and good test ability can be mined, and the potential testers can provide relatively good application test reports for application contractors so that the application contractors can carry out vulnerability patch on applications.

The invention discloses a text description generating method, system and device of a crowdsourcing test report. The method comprises the steps that screenshots and text descriptions in the test reportare acquired, key word sequences used for embodying test information in the text descriptions are determined, and hierarchical clustering is conducted on the screenshots according to a preset hierarchical clustering standard to obtain a plurality of sets of screenshot class clusters; the key word sequences which correspond to the test report and to which each screenshot in each set of screen capture class cluster belongs are input to a probability language model, and a class cluster text description generated by the probability language model is obtained; according to a preset classificationstrategy, the screenshot class cluster to which each to-be-diagnosed screenshot belongs in the to-be-diagnosed test report is determined, and a class cluster text description corresponding to the screenshot class cluster to which the to-be-diagnosed screenshot belongs is obtained. The number of the screenshots is large, the obtained text descriptions are rich, a developer can understand the test report, and the efficiency of diagnosing and understanding the test report is improved for the developer; the screenshots are objective, so that the diagnosis and understanding of the developer are noteasy to mislead.

The invention discloses a mobile application crowdsourcing test report automatic evaluation method and a computer storage medium, and the method comprises the following steps: 1), inputting a test report set and the historical credibility of a worker, removing an invalid test report, and carrying out the word segmentation and stop word removal of the remaining test report set; 2) clustering the test reports according to the discovered defects to form a plurality of types of defect test reports, and selecting the grade of the maximum proportion as the defect grade of the type of defect test reports; 3) constructing a plurality of normativity indexes and corresponding step type metric functions, evaluating the test report, and converting the evaluation into a normativity score of the test report; 4) according to the defect grade and the normativity score, obtaining the final score of the test report. The defect that an existing crowdsourcing test report quality evaluation method lacks test report content quality evaluation is overcome, and the overall performance of a crowdsourcing test platform is improved.

The disclosure provides a method and an electronic device for recommending crowdsourced tester and crowdsourced testing. The method comprises: collecting a requirement description of a crowd testing task at a time point in a process of crowd software testing and historical crowd testing reports of each tester to be recommended; obtaining a process context and a resource context of each tester to be recommended; inputting the extracted features into a learning to rank model to obtain an initial ranking list of the recommended testers, and re-ranking the initial ranking list based on diversity contributions of expertise and device to obtain a final ranking list. The disclosure can more accurately recommend testers to take accuracy and diversity of the recommended testers into consideration, so that the testers can be dynamically planned during the crowd testing to improve the bug detection rate, shorten the completion cycle of the crowd testing task.

The invention discloses a web service crowdsourcing test task allocation method based on deep reinforcement learning, which comprises the steps of a deep reinforcement learning-based Web service testtask allocation model is trained according to data information in a worker pool and a task pool on a crowdsourcing platform; the crowdsourcing platform receives a test task submitted by a demander, and allocates the test task by using the trained Web service test task allocation model; and a worker receives and executes the task and feeds back a task test result to the crowdsourcing platform, andthe crowdsourcing platform transmits the test result fed back by the worker to the corresponding task demander. A deep reinforcement learning method DQN is used to train a Web service test task allocation model. The effect of distributing the Web service crowdsourcing test tasks in real time is achieved, it can be guaranteed to a certain extent that the test tasks can be handled by appropriate test personnel on the crowdsourcing test platform, and the test effect is improved.

The invention discloses a voice recognition system crowdsourcing test case generation method based on test requirements. The method is characterized in that a provided voice recognition system is tested based on a crowdsourcing technology, the test requirements submitted by a task requester are recognized and extracted, a voice recognition keyword list in a preset keyword library is used for filtering out texts related to test case content from recognized text information fragments, and the texts serve as candidate test requirements. Three parts are included: a voice recognition test parametercollection module, a text extraction module and a test case generation module. Input is a text test demand, feature extraction is performed based on a user demand, and a test template is generated. The method has the following beneficial effect that the multi-dimensional and diversified requirements of ASR testing are met. The test participation threshold of crowdsourcing workers is reduced, testtasks are not limited by places and identities, and the crowdsourcing workers can sign up to participate in the test as long as the crowdsourcing workers have test tools and test capabilities, so that the test loss is reduced, and the product research and development period is shortened.

The invention provides a crowdsourcing test task selection method and an electronic device, and the crowdsourcing test task selection method comprises the steps: collecting task demand description andcorresponding crowdsourcing reports of n open crowdsourcing test tasks, and calculating each descriptive term vector of each open crowdsourcing test task; extracting task selection features and inputting the task selection features into a task selection model through acquired activity information of a crowdsourcing test person and preference information and specialty information of each descriptive term vector of an i-th open crowdsourcing test task, and predicting the defect discovery probability of the crowdsourcing test person in each open crowdsourcing test task, i being greater than or equal to 1 and less than or equal to n; and selecting an optimal crowdsourcing test task based on the probability of discovering the defects. According to the crowdsourcing test task selection method,task information and crowd-testing personnel information on the crowd-testing platform are better represented; proper tasks are selected for crowd-testing personnel by mining and modeling historical records of the personnel; the task selection and exploration time of the personnel is shortened; the defect discovery efficiency of the personnel is improved; and the income of the personnel on the crowd-testing platform is improved.

The invention discloses a crowdsourcing system-oriented simulation test case generation method and device, and the method comprises the steps: firstly inputting a sample obtained by a crowdsourcing platform into a VAE encoder, and learning a mean value and a variance of worker capability and task difficulty labeled for a real sample, namely obtaining the normal distribution of the corresponding worker capability and task difficulty; meanwhile, generating Gaussian mixture distribution of the capabilities and task difficulties of all workers participating in labeling; secondly, obtaining hidden variables of the VAE by using a reconstruction technique, inputting the hidden variables into a decoder of the VAE, and training the decoder by using gradient descent; and finally, inputting the worker ability and task difficulty randomly generated by Gaussian mixture distribution into a trained decoder, and generating a large number of crowdsourcing test cases which are independent and identically distributed with a real sample. According to the method, the neural network is combined to solve the problem that the crowdsourcing test cases are few and the coverage is narrow due to the cost, so that the result convergence method is difficult to comprehensively evaluate, and the crowdsourcing result convergence method can be comprehensively and effectively evaluated.

The invention relates to a device and method for processing running contexts of an application in a crowdsourcing test. The device comprises a crash monitoring module, an interaction module, a compression module and an upload module, wherein the crash monitoring module is configured to detect the crash of the application; the interaction module is configured to detect the selection of a tester to the active feedback by receiving the input of the tester; the compression module is configured to compress the behavior information of the recorded running contexts in response to the crash of the application which is detected or the mode that the tester selects active feedback so as to obtain the running contexts to be submitted; the upload module is configured to upload the running contexts to be submitted to a test platform; the step of compressing the behavior information in the recorded running contexts comprises the process of deleting the description of at least one event from the running contexts and simplifying at least one of the running contexts based on the repeated content in the running contexts; and the recorded running contexts contain activities related to the application and the behavior information of the event that occurs in the activities.

The present disclosure provides an information processing apparatus, an information processing method, and an information processing device for crowdsourcing tests. The information processing apparatus comprises: an extraction unit, for extracting test scenarios from a test record stored in a crowdsourcing test database, wherein the test scenario comprises an operation sequence of crowdsourcing testers; a clustering unit, for, according to the operation sequence of the crowdsourcing testers, clustering the extracted test scenarios to obtain a plurality of test scenario categories; and a determination unit, for determining the importance of each test scenario category to obtain test cases based on the importance. According to the information processing apparatus, the information processing method, and the information processing device provided by the present disclosure, at least important test cases can be effectively selected, so that the efficiency of crowdsourcing tests can be improved.

The invention provides a crowdsourcing tester recommendation and crowdsourcing test method and an electronic device. The crowdsourcing tester recommendation and crowdsourcing test method comprises thesteps of: collecting crowdsourcing task demand description at a time point in a crowdsourcing software test process and historical crowdsourcing reports of to-be-recommended personnel; acquiring process context and resource context of each to-be-recommended personnel; and inputting the extracted features into a sorting learning model to obtain an initial recommended personnel sorting, and resorting the initial recommended personnel sorting based on specialty and equipment diversity contribution degrees to obtain a final recommended personnel sorting. Personnel recommendation is performed moreaccurately based on the current context information, and the recommended personnel can give consideration to both accuracy and diversity, so that the personnel can be dynamically planned in the crowddetection process, the defect detection rate is improved, the completion period of the crowd detection task is shortened, and the formation of a more efficient crowd detection service mode is promoted.

An Android crowdsourcing test feedback method based on astep prompt is characterized in that a user test behavior database is established through recording a test process by a driver of a crowdsourcing test platform Kikbug; in the test process, the test step information of the current test personnel is collected and compared with the recorded and triggered abnormal operation information in the database, so that the problem that a crowdsourcing test report is not professional enough is solved, and the test personnel are guided to complete a more efficient crowdsourcing test. The method comprises the steps that (1) establishing an abnormal test behavior database, completing all to-be-tested tasks through a task provider, and initializing the database; (2) comparing the current tester with anoperation sequence in a database, and guiding the tester to trigger an abnormity; (3) storing new abnormities appearing in the test process; (4) when the reproduction frequency of a certain abnormityexceeds a threshold value, defining the abnormity as a confirmed abnormity, and not guiding a tester to verify the abnormity any more; and (5) feeding back the abnormity to the task provider.

The invention discloses a test coverage calculation method and device based on an execution path tracking technology. The method comprises the following steps: collecting change data when a user triggers a control; sorting the change data according to the order of responding to the user triggering controls to obtain a sorted chart; counting execution paths based on the response chart, and performing gathering to generate a test execution path event set; and respectively calculating the test coverage rate and the trigger proportion of each execution path by adopting the test execution path event set. According to the method and device, test execution paths of different crowdsourcing workers can be tracked, and respective test coverage rates of the crowdsourcing workers can be counted and calculated and are fed back to the crowdsourcing workers in real time, so that the crowdsourcing workers can better understand the test defects of the crowdsourcing workers, and then more sufficient testing is carried out to ensure test coverage in crowdsourcing testing; and meanwhile, according to the triggering proportion of each event, a developer can better find the problem and the part in which people are interested, so that the mobile application is further perfected.

The invention discloses a Chinese crowdsourcing test report clustering method based on semantic similarity, and the method comprises the steps: inputting a Chinese test report set, removing invalid test reports, and carrying out the sentence segmentation of effective test reports, and obtaining a test report sentence pair data set; constructing a test report sentence pair training set to train the semantic similarity model to obtain a semantic similarity calculation model; inputting the test report sentence pair data set into a semantic similarity calculation model for semantic similarity calculation to obtain a test report similarity matrix; setting an expected bug number of the test item, and performing spectral clustering according to the test report similarity matrix to obtain a test report class cluster; and decomposing the test report similarity matrix according to the test report class clusters to obtain the test report similarity matrix of each class cluster, and calculating a test report with an accumulated similarity score Top-5 in each class cluster as a final output result. The test report review automation process of the crowdsourcing test platform is improved, and the test report review efficiency is effectively improved.

The invention provides a cooperative crowdsourcing test oriented quality control method, and realizes a cooperative crowdsourcing test oriented quality control system on the basis of the cooperative crowdsourcing test oriented quality control method. The method is characterized in that 1, each operation of the crowdsourcing workers is evaluated and fed back in real time, so the quality awareness of the crowdsourcing workers is improved, and the disastrous behaviors of the crowdsourcing workers are recorded; 2, the quality grade of the report is determined through automatic scoring of the report, and a favorable reference is provided for managers to identify effective reports; and 3, a report auditing function is provided, so that a manager can conveniently check all information of the report and quickly and accurately auditing the report. The invention has the beneficial effects that the quality consciousness of crowdsourcing workers is improved through various designed quality controlmethods, the ratio of effective Bug reports in test tasks is improved, and managers are helped to identify the Bug reports and the quality of the crowdsourcing workers more quickly and accurately.

The embodiment of the invention provides a crowdsourcing test report recommendation method and device, and electronic equipment. The method comprises the following steps: obtaining a first defect report and a plurality of historical defect reports, determining a first feature vector of the first defect report and a second feature vector of each historical defect report, determining the similarity between the first feature vector and each second feature vector, and sorting the similarities; the historical defect reports are recommended on the basis of the sorting result, whether the defect reports similar to the first defect report exist in the historical defect reports or not can be determined, and therefore the defect reports with repeated content are prevented from occurring in the process that a user edits the first defect report, and the accuracy of the crowdsourcing test task is improved.

The invention provides a cooperation method for a crowdsourcing test platform, and the method has a reasonable guide mechanism and a real-time similar report recommendation mechanism, so crowdsourcingworkers carry out crowdsourcing software testing in a cooperation mode. The method is characterized in that the method comprises the steps: 1, analyzing the page structure of an application to be tested, enabling a defect report to carry a page label, and guiding the crowdsourcing workers to test by displaying page coverage of defects through Echarts Graph; 2, recommending similar defect reportsin real time through defect report label screening and text similarity calculation. 3, adding the user cooperation such as Fork modification, like clicking and treading of the defect reports, so thatthe defect reports are divided into a single type and a tree type, and crowdsourcing workers can audit the defect reports. The method has the beneficial effects that the test path is visualized through the page structure of the to-be-tested application, crowdsourcing workers are guided to reasonably test, and the test coverage rate is increased; through similar report recommendation, the number ofrepeated reports is reduced; and through a user cooperation mode, the quality of the defect report is iteratively improved.

The invention discloses a fairness-oriented crowdsourcing tester recommendation method and device. The method comprises the following steps: representing a demand description of a crowdsourcing task as a descriptive term vector; according to the descriptive term vector and the attribute information of the to-be-recommended personnel, predicting the defect detection probability of each to-be-recommended personnel in the crowdsourcing task; and setting a plurality of fairness targets including maximization of the defect discovery probability and minimization of the recommendation frequency difference, and performing multi-target optimization to obtain a crowdsourcing tester recommendation result. According to the method, the unfairness problem can be relieved, more fair recommendation is generated, and on the basis of generating the more fair recommendation, the defect detection efficiency is not influenced, and the defect detection efficiency is improved.

The invention discloses a crowdsourcing test report clustering method based on an AP affinity propagation algorithm. According to the method, firstly, keywords are extracted by utilizing a natural language processing technology, and identical or similar words are replaced by the same word by utilizing a synonym forest, so that the semantic deviation problem caused by different words is effectively solved. Then an asymmetric similarity strategy is adopted to calculate the similarity between test reports, and the strategy can effectively solve the problem that the similarity calculation is inaccurate under the condition that the length difference of the test reports is large. Finally, the test reports are clustered by using an AP affinity propagation clustering algorithm. Through the method, a crowdsourcing test report clustering effect can be improved, a developer only needs to select one test report from each cluster for examination, the examination time of the developer is greatly shortened, and the method has very important significance on software maintenance.

The invention discloses a crowdsourcing test task completion degree evaluation method and system. The method comprises the following steps: obtaining a test cost element data set according to test process conditions; performing correlation analysis on the test manpower cost data set and the test report cost data set; obtaining a defect data set of a test report result, and selecting a corresponding modeling equation from a pre-constructed segmented reliability model framework based on the correlation degree of a test manpower cost data set and a test report cost data set according to a correlation analysis result, estimating parameters of the modeling equation by utilizing a least square and maximum likelihood estimation method based on the defect data set to obtain the total number of potential defects of the software; according to the actual detection defect number of the defect data set and the total number of the potential defects of the software, the detection defect coverage rate is obtained through calculation, and the task completion degree is evaluated. The method has the advantages that the quality of test software is guaranteed, waste of a large amount of test time and test resources can be avoided, and the overall performance of a crowdsourcing test platform is improved.

The invention discloses a crowdsourcing test task allocation method based on student ability matching, which depicts student ability based on multiple dimensions and quantitatively evaluates the student ability level of each dimension through a fuzzy analytic hierarchy process. A test task attribute is taken as a demand guide to assign a weight of each dimension capability level, and a comprehensive capability matching value is calculated to serve as a matching basis for distributing crowdsourcing test tasks, so that a test platform can formulate an accurate and reasonable crowdsourcing test distribution strategy, the efficiency of the crowdsourcing test tasks is effectively improved, the quality of a test report is guaranteed, and the purposes of generating industry and education deep cooperation, providing professional talents for the industry and improving talent training quality for colleges and universities are achieved.

A crowdsourcing test report similarity detection method based on natural language processing adopts a natural language processing technology to detect the similarity of complex test reports submitted by crowdsourcing workers, and has the function of performing Chinese word segmentation, stop word removal and other preprocessing on the crowdsourcing reports. Sentences represented by the preprocessed phrases are represented as word vectors by utilizing a Word2Vec technology, a cosine similarity measurement mode is selected to calculate the distance between the word vectors, training is performed by adopting a semantic model trained according to a large amount of previous crowd test report data, and then each word vector is taken as input of K-Means clustering analysis, and clustering analysis is carried out on each word vector, and similar reports are classified into the same class according to a set similarity threshold, so that the similarity between the crowdsourcing test reports can be accurately measured.

The present invention provides a crowdsourcing testing method, comprising: the test client sends the characteristic information of the test client to the test server, and the test server selects the service to be tested according to the received characteristic information of at least one test client The client to be tested, the test server sends a list of services to be tested to the client to be tested, and the client to be tested tests the services to be tested in the list of services to be tested to obtain the service to be tested test results, and finally, the client to be tested sends the test results to the test server. This crowdsourcing testing method for testing network application services not only significantly improves the testing effect, but also effectively reduces the testing overhead.

A crowdsourcing test report fusion method based on public and private information mining comprises the steps that (1) feature mapping is carried out, each mode comprises public information shared by all modes and private information of the modes, and the modes extracted from an original feature space are mapped into public vectors and private vectors of the same dimension; (2) public information shared by all the modes included in different modes and private information of each mode are obtained; 3) modal relationship mining is carried out, and the obtained d-dimensional public information Ci and private information Sim are sequentially input into the LSTM for processing; and 4) label prediction is carried out, in multi-label learning, each mode has a specific contribution to multi-label prediction, and the relationship between labels is obtained by using the information of the different modes stored in an LSTM memory unit. According to the method, the low-dimensional common representation of all modes can be learned, the distinguishing information of each mode is extracted through forced orthogonal constraint, meanwhile, the common representation serves as a new mode to be added into a specific mode sequence, and label correlation mining based on the extracted mode sequence is achieved.

A method and device for processing crowdsourced test data, the method comprising: acquiring task description information of an application to be tested, the task description information being used to instruct crowdsourcing testing of the application to be tested; Set and the task description information, select the target tester set to test the application to be tested from the set of testers to be selected; generate a crowdsourced test strategy according to the set of target testers, and The application to be tested is delivered to each tester in the target tester set. By adopting this solution, the efficiency of selecting testers in the target tester set can be improved, and potential testers with good test resources and good testing ability can be discovered, and these potential testers can provide better application test reports for application developers. , so that the application developer can patch the application's vulnerabilities.

The invention provides a crowdsourcing tester recruitment method based on a social network. The crowdsourcing tester recruitment method comprises four stages of task segmentation, task release and propagation, propagation network construction and income distribution. The method comprises: firstly, dividing a public test task into small tasks which can be completed by a single tester; publishing the task to a crowdsourcing test platform, so that the task information is propagated in the social network; secondly, recording the superior of each tester by adopting an invitation code technology soas to form a propagation network; and finally, in a revenue distribution stage, performing revenue distribution according to the propagation network. According to the invention, the problem that enough testers cannot be recruited in a short time in the traditional crowdsourcing test is solved, and personnel guarantee is provided for crowdsourcing test. According to the method, the powerful personnel recruitment capability of the social network is utilized, enough workers are recruited for public survey in a short time, and meanwhile, a flexible income distribution mode is created, so that personnel recruitment has good motivation.

A multi-path recall recommendation method for crowdsourcing software testing comprises the steps: S1, conducting data collection, and obtaining a data set for multi-path recall, wherein the data set comprises user information, user social information, project information, project text description and user historical project completion conditions; S2, preprocessing the data, and calculating an input number for a multi-path recall model; S3, performing project multi-path recall, finding similar users of a target user by calculating user similarity, and finding historical completion projects of the target user and the similar users through a user-project interaction matrix; S4, performing user multi-path recall, finding similar items of a target item by calculating the item similarity and the item text embedding similarity, finding users completing the target item and the similar items through a user-item interaction matrix, and finding similar users of the users by calculating the user similarity and a user relation matrix. Thus, multi-path recall of crowdsourcing test users is completed.