Application control labeling method and device and computing equipment

Abstract

The invention discloses an application control labeling method and device and computing equipment, and the method is executed in the computing equipment and comprises the steps: recording an operation video containing one or more to-be-labeled controls; performing framing processing on the operation video to obtain a plurality of frames of pictures, and obtaining a plurality of pictures of different scenes as to-be-labeled pictures; for each to-be-labeled picture, determining position information of a to-be-labeled control in the to-be-labeled picture based on the control template; and generating annotation data of the to-be-annotated pictures on the basis of the position information so as to generate a training data set on the basis of the annotation data of the to-be-annotated pictures, and training an intelligent identification control model on the basis of the training data set. According to the technical scheme, the labor cost and the time cost of data labeling are saved, and the quality of the finally generated training data is improved.

Description

technical field [0001] The invention relates to the technical field of automated testing, in particular to a labeling method, labeling device and computing equipment for application controls. Background technique [0002] The intelligent recognition model in the field of automated testing aims to solve the problem of element positioning failure caused by fine-tuning of desktop elements in the image recognition coordinate positioning scene. At present, most of the machine learning and deep neural network algorithms required for intelligent recognition rely on training data. It is necessary to label a large number of picture materials to obtain a large amount of training data, which can then be used to train the recognition model. Due to the diversity and complexity of desktop application functions and the huge amount of control data, a large number of picture materials need to be marked. [0003] In the prior art, manual labeling is usually used to obtain the required traini...

AI Technical Summary

Problems solved by technology

There are certain defects in the use of manual data labeling. On the one hand, intelligent recognition training requires a large amount of training data, and data standards require a lot of manpower and time, resulting in high costs for producing training data.

On the other hand, human factors such as different subjective consciousness of the annotators, or different naming of the same controls, will lead to errors in the annotated data and low quality, which will affect the accuracy of the training results.

Images