Automatic evaluation system and method for video analysis algorithm

Abstract

The invention discloses an automatic evaluation system and method for a video analysis algorithm, and provides a computing power automatic evaluation module which can generate an evaluation template according to different types of algorithm evaluation rules and realize automatic evaluation of algorithm indexes such as performance, cost and perception of the algorithm according to the evaluation template. According to the method, the limitation of a traditional algorithm evaluation method is broken through, the quick evaluation problem of different manufacturers under the same kind of algorithms is solved, automatic evaluation can be carried out on the same kind of algorithms of multiple manufacturers according to a unified rule template, and the similar algorithms are visually embodied in the form of reports and charts. The method is suitable for a real-time and continuous intelligent video algorithm evaluation scene and is convenient to use, and a user can realize automatic evaluation of the intelligent video algorithm by customizing algorithm evaluation rule setting.

Description

technical field [0001] The present invention relates to the technical field of AI algorithms, in particular to an automatic evaluation system and method for video analysis algorithms. Background technique [0002] With the country's massive investment in and support for technology, the AI ​​market has gone through a third upsurge, from the original "symbolicism" to "machine learning, neural network" to the current "artificial intelligence (big data, chips, algorithm)” deep learning. Therefore, there are also a large number of algorithm manufacturers in the market that provide algorithms for various scenarios (face algorithm, vehicle algorithm, image recognition algorithm, etc.). Before the algorithm enters the market, each manufacturer will evaluate and test its algorithm to ensure the algorithm Performance, but the scope of evaluation is limited. The evaluation results provided by different manufacturers of the same algorithm, due to different evaluation methods and other ...

AI Technical Summary

Problems solved by technology

Therefore, there are also a large number of algorithm manufacturers in the market that provide algorithms for various scenarios (face algorithm, vehicle algorithm, image recognition algorithm, etc.). Before the algorithm enters the market, each manufacturer will evaluate and test its algorithm to ensure the algorithm Performance, but the scope of evaluation is limited. The evaluation results provided by different manufacturers of the same algorithm, due to different evaluation methods and other factors, lead to low comparability of evaluation results between algorithms

In short, traditional algorithm evaluation needs to be evaluated one by one, which is inefficient and the evaluation methods are not uniform

[0003] At present, there is a lack of effective management of algorithm evaluation methods and unified formulation of evaluation rules. The accuracy of the evaluation results obtained by the same algorithm in different evaluation rule systems will vary.

In addition, the evaluation results can only be obtained after going through the manufacturer or cumbersome procedures, which increases the difficulty of the early research, and the evaluation results cannot be achieved with evidence.

Images