Video object detection evaluation method, device, product and storage medium

Abstract

The present application relates to target detection technology, and discloses a method for evaluating video target detection, including calculating the intersection-over-union (IOU) value of the target on the video data source; and calculating the average precision (AP) value through the IOU value. Compared with the traditional method of only using target detection for images, this method can more completely demonstrate and evaluate the performance of the target detection algorithm on the video stream task. The present application also discloses a video target detection evaluation device, product, and storage medium.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, for example, to a video target detection and evaluation method, device, product and storage medium. Background Art

[0002] At present, object detection, as one of the most important computer vision tasks, has been widely used in various aspects of social production and life. Evaluating the accuracy and efficiency of object detection algorithms has become an urgent problem for developers to solve.

[0003] Existing object detection evaluation methods usually use images as the main data source and apply the detection algorithm to image datasets, such as COCO and VOC datasets.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art:

[0005] In practical applications, the data source of the target detection algorithm is generally a video stream. If it is evaluated based on image datasets, it is often easy to ignore many characteristics and details of the video stream, and it is impossible to fully display and evaluate the performance of the target detection algorithm on the video stream task. Summary of the invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The embodiments of the present disclosure provide a video object detection and evaluation method device, product and storage medium to solve the technical problems of multi-scenario applications.

[0008] In some embodiments, the method comprises:

[0009] Calculate the intersection-over-union (IOU) value of the target on the video data source;

[0010] The average precision AP value is calculated using the IOU value.

[0011] Specifically, the method for calculating the value of the intersection-over-union (IOU) of the target on the video data source includes:

[0012] Set the number of video frames on the video data source to Q;

[0013] The target IOU value is calculated according to formula (1), which is:

[0014]

[0015] Where o is the target in the video data source, k is the serial number of the frame in the video data source, Q is a natural number greater than 1, k is a natural number greater than or equal to 1 and less than Q-1, and B g (o,k) is the box area of ​​the real position of the target o in the kth frame, B r (o,k) is the box area of ​​the predicted position of target o in the kth frame obtained by the target detection algorithm.

[0016] Specifically, the method for calculating the value of the average precision AP by the IOU value includes:

[0017] Set the IOU threshold to h, and the value range of h is [0,1];

[0018] Determine whether the target is a positive sample or a negative sample according to the threshold;

[0019] Calculate the precision P and recall R of the target according to formula (2) and formula (3);

[0020] According to different h values ​​and the precision P and recall R corresponding to the h values, the PR curve is obtained, with Precision as the ordinate and Recall as the abscissa;

[0021] Performing an interpolation operation on the PR curve, that is, dividing the PR curve into n intervals according to the Recall value on the horizontal axis and obtaining the maximum value of the Precision value on the vertical axis in each of the intervals, and using the maximum value as the Precision value of each of the intervals;

[0022] Calculate the average precision AP value according to formula (4);

[0023] Among them, formula (2), formula (3) and formula (4) are respectively:

[0024] P=TP / (TP+FP) (2),

[0025] R = TP / (TP+FN) (3),

[0026]

[0027] TP stands for True Positive, which means that the target is judged as a positive sample and is actually a positive sample. FP stands for False Positive, which means that the target is judged as a positive sample but is actually a negative sample. FN stands for False Negative, which means that the target is judged as a negative sample but is actually a positive sample. i and r i+1 is the Recall value that divides the PR curve into several intervals, i is a natural number greater than or equal to 1, pinterp (r i+1 ) is [r i ,r i+1 ] is the maximum value of the Precision value on the vertical axis in the interval, where n is a natural number greater than or equal to 1.

[0028] Specifically, the method for determining whether the target is a positive sample or a negative sample according to the threshold includes:

[0029] If the IOU value of the target is greater than or equal to k, the target is judged to be a positive sample in the video data source, that is, TP; otherwise, the target is judged to be a negative sample in the video data source, that is, FP.

[0030] In some embodiments, the apparatus comprises:

[0031] The IOU calculation module is configured to calculate the intersection-over-union (IOU) value of the target on the video data source;

[0032] The AP calculation module is configured to calculate the value of the average precision AP according to the IOU value.

[0033] In some embodiments, the apparatus comprises:

[0034] A processor and a memory storing program instructions, wherein the processor is configured to execute the above-mentioned video target detection and evaluation method when running the program instructions.

[0035] In some embodiments, the product comprises:

[0036] The video object detection and evaluation device as described above.

[0037] In some embodiments, the storage medium stores program instructions, and when the program instructions are run, the video object detection and evaluation method described above is executed.

[0038] The video target detection evaluation method, device, product and storage medium provided by the embodiments of the present disclosure calculate the IOU value of the target on each frame of the video data stream and then obtain the average precision AP by interpolating the PR curve. Therefore, the detection effect of the target detection algorithm on the target in the video data stream can be known according to the AP value. Compared with the traditional method of only using target detection for pictures, it can more completely display and evaluate the performance of the target detection algorithm on the video stream task.

[0039] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0041] Figure 1 is a schematic diagram of a video object detection and evaluation method provided by an embodiment of the present disclosure; FIG.

[0042] 2 is a schematic diagram of a video object detection and evaluation device provided by an embodiment of the present disclosure;

[0043] Figure 3 is a schematic diagram of another video object detection and evaluation device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0045] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0046] Unless otherwise stated, the term "plurality" means two or more.

[0047] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0048] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0049] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0050] Combination Figure 1As shown, the embodiment of the present disclosure provides a video target detection and evaluation method, including:

[0051] S01, calculating the intersection-over-union (IOU) value of the target on the video data source;

[0052] S02: Calculate the average precision AP value using the above IOU value.

[0053] Optionally, in step S01, the method for calculating the value of the intersection-over-union (IOU) of the target on the video data source includes:

[0054] Set the number of video frames on the above video data source to Q;

[0055] The target IOU value is calculated according to formula (1), which is:

[0056]

[0057] Where o is the target in the video data source, k is the serial number of the frame in the video data source, Q is a natural number greater than 1, k is a natural number greater than or equal to 1 and less than Q-1, and B g (o,k) is the box area of ​​the real position of the target o in the kth frame, B r (o,k) is the box area of ​​the predicted position of target o in the kth frame obtained by the target detection algorithm.

[0058] Dividing the video data source into several frames and calculating the IOU value on the video data stream can more completely demonstrate and evaluate the performance of the target detection algorithm on the video stream task compared to the traditional method of only using target detection on images.

[0059] Optionally, the method for calculating the average precision AP value using the IOU value includes:

[0060] Set the IOU threshold to h, and the value range of h is [0,1];

[0061] Determine whether the target is a positive sample or a negative sample according to the threshold;

[0062] Calculate the precision P and recall R of the target according to formula (2) and formula (3);

[0063] According to different h values ​​and the precision P and recall R corresponding to the h values, the PR curve is obtained, with Precision as the ordinate and Recall as the abscissa;

[0064] Performing an interpolation operation on the PR curve, that is, dividing the PR curve into n intervals according to the Recall value on the horizontal axis and obtaining the maximum value of the Precision value on the vertical axis in each of the intervals, and using the maximum value as the Precision value of each of the intervals;

[0065] Calculate the average precision AP value according to formula (4);

[0066] Among them, formula (2), formula (3) and formula (4) are respectively:

[0067] P=TP / (TP+FP) (2),

[0068] R = TP / (TP+FN) (3),

[0069]

[0070] TP stands for True Positive, which means that the target is judged as a positive sample and is actually a positive sample. FP stands for False Positive, which means that the target is judged as a positive sample but is actually a negative sample. FN stands for False Negative, which means that the target is judged as a negative sample but is actually a positive sample. i and r i+1 is the Recall value that divides the PR curve into several intervals, i is a natural number greater than or equal to 1, p interp (r i+1 ) is [r i ,r i+1 ] is the maximum value of the Precision value on the vertical axis in the interval, where n is a natural number greater than or equal to 1.

[0071] By using interpolation calculation, the original PR curve is divided into several intervals, and the sum of the interval areas is calculated to obtain the AP value, which reduces the jitter of the original PR curve and obtains a smoother PR curve. Generally speaking, the higher the AP value, the better the target detection method is.

[0072] Optionally, the method for determining whether the target is a positive sample or a negative sample according to the threshold includes:

[0073] If the IOU value of the target is greater than or equal to k, the target is judged to be a positive sample in the video data source, that is, TP; otherwise, the target is judged to be a negative sample in the video data source, that is, FP.

[0074] Combination Figure 2 As shown, an embodiment of the present disclosure provides a video object detection and evaluation device, including an IOU calculation module 21 and an AP calculation module 22.

[0075] The IOU calculation module 21 is configured to calculate the value of the intersection-over-union (IOU) of the target on the video data source;

[0076] The AP calculation module 22 is configured to calculate the value of the average precision AP according to the IOU value.

[0077] Compared with the traditional method of only using target detection on pictures, the video target detection evaluation device provided by the embodiment of the present disclosure can more completely demonstrate and evaluate the performance of the target detection algorithm on the video stream task.

[0078] Combination Figure 3 As shown, an embodiment of the present disclosure provides a video target detection and evaluation device, including a processor 100 and a memory 101. Optionally, the device may also include a communication interface 102 and a bus 103. The processor 100, the communication interface 102, and the memory 101 may communicate with each other through the bus 103. The communication interface 102 may be used for information transmission. The processor 100 may call the logic instructions in the memory 101 to execute the video target detection and evaluation method of the above embodiment.

[0079] In addition, the logic instructions in the memory 101 described above may be implemented in the form of software functional units and when sold or used as independent products, may be stored in a computer-readable storage medium.

[0080] The memory 101 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 100 executes the function application and data processing by running the program instructions / modules stored in the memory 101, that is, the video object detection and evaluation method in the above embodiment is implemented.

[0081] The memory 101 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 101 may include a high-speed random access memory and may also include a non-volatile memory.

[0082] The embodiment of the present disclosure provides a product, including the above-mentioned video object detection and evaluation device.

[0083] The embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned video object detection and evaluation method.

[0084] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned video object detection and evaluation method.

[0085] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0086] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0087] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0088] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0089] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0090] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A video target detection and evaluation method, characterized in that: include: Calculate the intersection-over-union (IOU) value of the target on the video data source; Calculate the average precision AP value by the IOU value; The method for calculating the value of the intersection-over-union (IOU) of the target on the video data source includes: Set the number of video frames on the video data source to Q; The target IOU value is calculated according to formula (1), which is: Where o is the target in the video data source, k is the serial number of the frame in the video data source, Q is a natural number greater than 1, k is a natural number greater than or equal to 1 and less than Q-1, and B g (o,k) is the box area of ​​the real position of the target o in the kth frame, B r (o,k) is the box area of ​​the predicted position of target o in the kth frame obtained by the target detection algorithm; The method for calculating the value of the average precision AP by the IOU value includes: Set the IOU threshold to h, and the value range of h is [0,1]; Determine whether the target is a positive sample or a negative sample according to the threshold; Calculate the precision P and recall R of the target according to formula (2) and formula (3); According to different h values ​​and the precision P and recall R corresponding to the h values, the PR curve is obtained, with Precision as the ordinate and Recall as the abscissa; Performing an interpolation operation on the PR curve, that is, dividing the PR curve into n intervals according to the Recall value on the horizontal axis and obtaining the maximum value of the Precision value on the vertical axis in each of the intervals, and using the maximum value as the Precision value of each of the intervals; Calculate the average precision AP value according to formula (4); Among them, formula (2), formula (3) and formula (4) are respectively: P=TP / (TP+FP) (2), R=TP / (TP+FN) (3), TP stands for True Positive, which means that the target is judged as a positive sample and is actually a positive sample. FP stands for False Positive, which means that the target is judged as a positive sample but is actually a negative sample. FN stands for False Negative, which means that the target is judged as a negative sample but is actually a positive sample. i and r i+1 is the Recall value that divides the PR curve into several intervals, i is a natural number greater than or equal to 1, p interp (r i+1 ) is [r i ,r i+1 ] is the maximum value of the Precision value on the ordinate axis in the interval, where n is a natural number greater than or equal to 1; The method for determining whether the target is a positive sample or a negative sample according to the threshold comprises: If the IOU value of the target is greater than or equal to k, the target is judged to be a positive sample in the video data source, that is, TP; otherwise, the target is judged to be a negative sample in the video data source, that is, FP.

2. A video object detection and evaluation device, characterized in that: include: The IOU calculation module is configured to calculate the intersection-over-union (IOU) value of the target on the video data source; An AP calculation module is configured to calculate an average precision AP value through the IOU value; The calculation of the intersection-over-union (IOU) value of the target on the video data source includes: Set the number of video frames on the video data source to Q; The target IOU value is calculated according to formula (1), which is: Where o is the target in the video data source, k is the serial number of the frame in the video data source, Q is a natural number greater than 1, k is a natural number greater than or equal to 1 and less than Q-1, and B g (o,k) is the box area of ​​the real position of the target o in the kth frame, B r (o,k) is the box area of ​​the predicted position of target o in the kth frame obtained by the target detection algorithm; The calculation of the average precision AP value by the IOU value includes: setting The threshold of IOU is h, and the value range of h is [0,1]; Determine whether the target is a positive sample or a negative sample according to the threshold; Calculate the precision P and recall R of the target according to formula (2) and formula (3); According to different h values ​​and the precision P and recall R corresponding to the h values, the PR curve is obtained, with Precision as the ordinate and Recall as the abscissa; Performing an interpolation operation on the PR curve, that is, dividing the PR curve into n intervals according to the Recall value on the horizontal axis and obtaining the maximum value of the Precision value on the vertical axis in each of the intervals, and using the maximum value as the Precision value of each of the intervals; Calculate the average precision AP value according to formula (4); Among them, formula (2), formula (3) and formula (4) are respectively: P=TP / (TP+FP) (2), R=TP / (TP+FN) (3), TP stands for True Positive, which means that the target is judged as a positive sample and is actually a positive sample. FP stands for False Positive, which means that the target is judged as a positive sample but is actually a negative sample. FN stands for False Negative, which means that the target is judged as a negative sample but is actually a positive sample. i and r i+1 is the Recall value that divides the PR curve into several intervals, i is a natural number greater than or equal to 1, p interp (r i+1 ) is [r i ,r i+1 ] is the maximum value of the Precision value on the vertical axis in the interval, where n is a natural number greater than or equal to 1.

3. A video target detection and evaluation device, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the video object detection and evaluation method according to claim 1 when running the program instructions.

4. A video object detection and evaluation product, characterized in that: It comprises the video target detection and evaluation device as described in claim 2 or 3.

5. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the video object detection and evaluation method as claimed in claim 1 is executed.

Images