A robust performance testing method, device, electronic device, and storage medium

Abstract

The present application relates to a robust performance testing method, apparatus, electronic device, and storage medium, and relates to the field of intelligent driving technology. The method includes: constructing a metamorphic relationship of a multi-object tracking system based on the application scenario of the multi-object tracking system; obtaining a derived test case corresponding to a source test case based on the metamorphic relationship; inputting the source test case and the derived test case into the multi-object tracking system to obtain a source multi-object tracking result and a derived multi-object tracking result; and obtaining a robustness verification result of the multi-object tracking system based on the relationship between the source multi-object tracking result and the derived multi-object tracking result. By using this method, the robustness of the multi-object tracking system can be tested by verifying whether the metamorphic relationship is satisfied between the source multi-object tracking result and the derived multi-object tracking result, avoiding the need for a large amount of manual marking of expected output data frames in traditional testing methods and improving the testing efficiency of the robustness of the multi-object tracking system.

Description

Technical Field

[0001] The present application relates to the field of intelligent tracking technology, and particularly relates to a robust performance testing method, device, electronic device, and storage medium. Background Art

[0002] A multi-object tracking system is used to detect multiple objects in consecutive video frames and track their trajectories. With the development of artificial intelligence technologies such as deep learning, multi-object tracking systems have been widely applied in many safety-critical fields, such as autonomous driving, intelligent monitoring, behavior recognition, etc. The reliability of multi-object tracking systems is very important, especially their robustness in different scenarios. A multi-object tracking system with poor robustness is very dangerous and may lead to serious accidents. Therefore, before actual application, it is necessary to conduct sufficient robustness tests on multi-object tracking systems in different scenarios.

[0003] In traditional testing methods, the correctness of a program is verified by comparing the actual output with the expected output, which requires a series of test cases marked with expected outputs. For systems based on neural network models such as multi-object tracking systems, the annotation work of their datasets often consumes a lot of manpower. Taking the annotation work of videos as an example, a video contains multiple frames of pictures, each frame of picture contains multiple objects, and multiple objects such as vehicles on the road may be very similar to each other, which makes the annotation work very heavy and difficult. Therefore, traditional testing methods are not applicable to the robustness testing of multi-object tracking systems.

[0004] In addition, in the application scenarios of actual multi-object tracking systems, there are many critical edge scenarios, such as sudden camera failures, resulting in short-term video blurring. Such occasional edge scenarios are difficult to collect in real life, which also makes it difficult to achieve a comprehensive and effective test of the robustness of multi-object tracking systems. Summary of the Invention

[0005] The present application provides a robust performance testing method, device, electronic device, and storage medium to achieve a comprehensive and effective test of the robustness of multi-object tracking systems.

[0006] To achieve the above object, the present application provides the following solutions.

[0007] In a first aspect, the present application provides a robust performance testing method, and the method includes the following steps:

[0008] Based on the application scenario of the multi-object tracking system, construct a metamorphic relationship of the multi-object tracking system; the metamorphic relationship refers to the corresponding relationship between the source test case input to the multi-object tracking system and the derived test case;

[0009] Based on the metamorphic relationship, obtain the derived test case corresponding to the source test case;

[0010] Input the source test case and the derived test case into the multi - object tracking system to obtain the source multi - object tracking result and the derived multi - object tracking result;

[0011] Based on the relationship between the source multi - object tracking result and the derived multi - object tracking result, obtain the robustness verification result of the multi - object tracking system.

[0012] Furthermore, constructing the metamorphic relationship of the multi - object tracking system based on the application scenario of the multi - object tracking system includes the following steps:

[0013] When the application scenario of the multi - object tracking system is a scenario of simulating the acceleration of a tracked object, obtain a preset source video test case;

[0014] Extract one picture frame every K frames from the source video test case containing multiple picture frames to obtain a derived video test case corresponding to the source video test case;

[0015] Based on the mapping relationship between the source video test case and the derived video test case, construct the metamorphic relationship of the multi - object tracking system in the scenario of simulating the acceleration of a tracked object.

[0016] Furthermore, constructing the metamorphic relationship of the multi - object tracking system in the scenario of simulating the acceleration of a tracked object based on the mapping relationship between the source video test case and the derived video test case includes the following steps:

[0017] From the preset source video test case V = {f1, f2, …, f n}, extract the picture frames at a fixed interval K to obtain the derived video test case V′ = {f′1, f′2, …, f′ m};

[0018] Based on the source video test case V = {f1, f2, …, f n} and the derived video test case V′ = {f′1, f′2, …, f′ m}, construct the metamorphic relationship of the multi - object tracking system in the scenario of simulating the acceleration of a tracked object. The metamorphic relationship is as follows:

[0019] f′ i = f ik , i ∈ N +, ik ≤ n.

[0020] Furthermore, constructing the metamorphic relationship of the multi - object tracking system based on the application scenario of the multi - object tracking system includes the following steps:

[0021] When the application scenario of the multi-object tracking system is a scenario of simulating a camera failure, obtain a preset source video test case;

[0022] Randomly select a specific number of reference picture frames from the source video test case containing multiple picture frames, and use the previous picture frame of the reference picture frames to overwrite the reference picture frames to form a derivative video test case corresponding to the source video test case.

[0023] Based on the mapping relationship between the source video test case and the derivative video test case, construct the metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of a tracking object.

[0024] Further, the step of randomly selecting a specific number of reference picture frames from the source video test case containing multiple picture frames, and using the previous picture frame of the reference picture frames to overwrite the reference picture frames to form a derivative video test case corresponding to the source video test case includes the following steps:

[0025] Randomly select several reference picture frames {q1, q2,..., q n} from the preset source video test case V = {f1, f2,..., f r};

[0026] Use the previous picture frame of the reference picture frames to overwrite the reference picture frames {q1, q2,..., q r} to form a derivative video test case V' = {f'1, f'2,..., f' m} corresponding to the source video test case;

[0027] Based on the source video test case V = {f1, f2,..., f n} and the derivative video test case V' = {f'1, f'2,..., f' m}, construct the metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of a tracking object. The metamorphic relationship is as follows:

[0028]

[0029] Further, the step of constructing the metamorphic relationship of the multi-object tracking system based on the application scenario of the multi-object tracking system includes the following steps:

[0030] When the application scenario of the multi-object tracking system is a scenario of simulating a camera failure, obtain a preset source video test case;

[0031] Randomly select a specific number of reference picture frames from the source video test case containing multiple picture frames, and pixelate the reference picture frames to obtain a derivative video test case.

[0032] Based on the mapping relationship between the source video test case and the derived video test case, construct the metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of the tracked object.

[0033] Further, obtaining the robustness verification result of the multi-object tracking system based on the relationship between the source multi-object tracking result and the derived multi-object tracking result further includes the following steps:

[0034] If the source multi-object tracking result and the derived multi-object tracking result satisfy the metamorphic relationship, it indicates that the robustness of the multi-object tracking system is good.

[0035] In a second aspect, the present application provides a robustness performance testing device, and the device includes:

[0036] A metamorphic relationship construction module, which is used to construct the metamorphic relationship of the multi-object tracking system based on the application scenario of the multi-object tracking system; the metamorphic relationship refers to the corresponding relationship between the source test case and the derived test case input to the multi-object tracking system;

[0037] A derived use case acquisition module, which is used to obtain the derived test case corresponding to the source test case based on the metamorphic relationship;

[0038] A tracking result acquisition module, which is used to input the source test case and the derived test case into the multi-object tracking system to obtain the source multi-object tracking result and the derived multi-object tracking result;

[0039] A robustness verification module, which is used to obtain the robustness verification result of the multi-object tracking system based on the relationship between the source multi-object tracking result and the derived multi-object tracking result.

[0040] Further, the metamorphic relationship construction module includes:

[0041] A first source video acquisition sub-module, which is used to obtain a preset source video test case when the application scenario of the multi-object tracking system is a scenario of simulating the acceleration of the tracked object;

[0042] A first derived video acquisition sub-module, which is used to extract one picture frame every K frames from the source video test case including multiple picture frames to obtain the derived video test case corresponding to the source video test case;

[0043] A first construction sub-module, which is used to construct the metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of the tracked object based on the mapping relationship between the source video test case and the derived video test case.

[0044] Further, the construction sub-module includes:

[0045] A derivative use case generation unit, which is used to extract picture frames from a preset source video test case V={f1, f2, …, f n} containing multiple picture frames at a fixed interval K to obtain a derivative video test case V′={f′1, f′2, …, f′ m};

[0046] A metamorphic relationship generation unit, which is used to construct a metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of a tracked object based on the source video test case V={f1, f2, …, f n} and the derivative video test case V′={f′1, f′2, …, f′ m}, and the metamorphic relationship is as follows:

[0047] f′ i =f ik , i∈N+, ik≤n.

[0048] Further, the metamorphic relationship construction module includes:

[0049] A second source video acquisition sub-module, which is used to acquire a preset source video test case when the application scenario of the multi-object tracking system is a scenario of simulating a camera failure;

[0050] A second derivative video acquisition sub-module, which is used to randomly select a specific number of reference picture frames from a source video test case containing multiple picture frames, and use the previous picture frame of the reference picture frame to overwrite the reference picture frame to form a derivative video test case corresponding to the source video test case;

[0051] A second construction sub-module, which is used to construct a metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of a tracked object based on the mapping relationship between the source video test case and the derivative video test case.

[0052] Further, the second derivative video acquisition sub-module includes:

[0053] A reference frame acquisition unit, which is used to randomly select several reference picture frames {q1, q2, …, q n} from a preset source video test case V={f1, f2, …, f r};

[0054] An overwriting unit, which is used to use the previous picture frame of the reference picture frame to overwrite the reference picture frames {q1, q2, …, q r} Generate a derivative video test case V′ = {f′1, f′2, …, f′ corresponding to the source video test case m};

[0055] A metamorphic relationship generation unit, which is used to construct a metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of a tracked object based on the source video test case V = {f1, f2, …, f n} and the derivative video test case V′ = {f′1, f′2, …, f′ m}, and the metamorphic relationship is as follows:

[0056]

[0057] Further, the metamorphic relationship construction module includes:

[0058] A third source video acquisition sub-module, which is used to acquire a preset source video test case when the application scenario of the multi-object tracking system is a scenario of simulating a camera failure;

[0059] A third derivative video acquisition sub-module randomly selects a specific number of reference picture frames from the source video test case containing multiple picture frames, and pixelates the reference picture frames to obtain a derivative video test case.

[0060] A third construction sub-module, which is used to construct a metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of a tracked object based on the mapping relationship between the source video test case and the derivative video test case.

[0061] The beneficial effects brought by the technical solution provided by this application include:

[0062] In this application, the test server constructs a metamorphic relationship of the multi-object tracking system based on the application scenario of the multi-object tracking system; this metamorphic relationship refers to the correspondence relationship between the source test case and the derivative test case input by the multi-object tracking system; based on this metamorphic relationship, a derivative test case corresponding to the source test case is obtained; the source test case and the derivative test case are input into the multi-object tracking system to obtain the source multi-object tracking result and the derivative multi-object tracking result; based on the relationship between the source multi-object tracking result and the derivative multi-object tracking result, a robustness verification result of the multi-object tracking system is obtained. By using this method, the robustness of the multi-object tracking system can be tested by verifying whether the source multi-object tracking result and the derivative multi-object tracking result satisfy the metamorphic relationship, avoiding the need for a large amount of manual marking of the expected output data frames in the traditional test method, and improving the test efficiency of the robustness of the multi-object tracking system. Description of the Drawings

[0063] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0064] Figure 1 It is a flowchart of the steps of the robust performance test method provided in the embodiments of the present application. Specific embodiments

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0066] The following further elaborates on the embodiments of the present application with reference to the drawings.

[0067] The embodiments of the present application provide a robust performance test method, device, electronic device, and storage medium, which improve the test efficiency of the robustness of the multi-object tracking system.

[0068] The following further elaborates on the embodiments of the present application with reference to the drawings.

[0069] See Figure 1 As shown, the embodiments of the present application provide a robust performance test method, which includes the following steps:

[0070] S1. Based on the application scenario of the multi-object tracking system, construct the metamorphic relationship of the multi-object tracking system;

[0071] The multiple objects tracking system (MOTS) adopts advanced image detection, recognition, and tracking technologies, and cooperates with a precision motion control system to achieve continuous and rapid tracking and capture of multiple moving objects in a large scene. The test and video analysis functions of the multi-object tracking system are integrated in an independent system. By intelligently analyzing the video information collected by the front-end camera, it automatically collects and classifies data on abnormal behaviors and events and triggers alarms in a linked manner. At the same time, the analysis data and video recordings can be viewed in real time in the background, and video extraction and evidence collection can be performed through event retrieval after the event.

[0072] The so-called "robustness" refers to the property of a control system to maintain certain performance under parameter perturbations of a certain (structure, size). According to different definitions of performance, it can be divided into stability robustness and performance robustness; the metamorphic relationship means

[0073] In traditional robust performance testing methods, usually the actual output test cases are compared with the expected output test cases and marked, so a large amount of manual marking is required, which reduces the testing efficiency of the robustness of the multi-object tracking system.

[0074] The multi-object tracking system has a wide range of application scenarios. For example, when the tracked object suddenly accelerates during driving, when the tracked object is blocked by an obstacle resulting in a decrease in pixels, and when the tracking image frame is lost during tracking.

[0075] Specifically, when the application scenario of the multi-object tracking system is a scenario of simulating the acceleration of a tracked object, the test server obtains a preset source video test case, extracts one picture frame every K frames from the source video test case containing multiple picture frames, and obtains a derivative video test case corresponding to the source video test case; in one embodiment, it can also be randomly selecting a specific number of reference picture frames from the source video test case containing multiple picture frames and pixelating the reference picture frames to obtain a derivative video test case.

[0076] The test server constructs a metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of the tracked object based on the mapping relationship between the source video test case and the derivative video test case.

[0077] S2. Input the source test case and the derivative test case into the multi-object tracking system to obtain the source multi-object tracking result and the derivative multi-object tracking result.

[0078] The test case is used to test the robustness of the multi-object tracking system and is usually composed of video frames.

[0079] Specifically, input the source test case into the multi-object tracking system to obtain the source multi-object tracking result, and input the derivative test case into the multi-object tracking system to obtain the derivative multi-object tracking result.

[0080] S3. Based on the relationship between the source multi-object tracking result and the derivative multi-object tracking result, obtain the robustness verification result of the multi-object tracking system.

[0081] If the metamorphic relationship is satisfied between the source multi-object tracking result and the derivative multi-object tracking result, it indicates that the robustness of the multi-object tracking system is good.

[0082] Specifically, the test server obtains a preset source video test case, extracts one picture frame every K frames from the source video test case containing multiple picture frames, obtains a derivative video test case corresponding to the source video test case, and then compares the consistency between the source multi-object tracking result and the derivative object tracking result. If they are consistent, it indicates that the multi-object tracking system has good robustness.

[0083] In this application, the test server constructs a metamorphic relationship of the multi-object tracking system based on the application scenario of the multi-object tracking system; this metamorphic relationship refers to the corresponding relationship between the source test case and the derivative test case input to the multi-object tracking system; based on this metamorphic relationship, a derivative test case corresponding to the source test case is obtained; the source test case and the derivative test case are input into the multi-object tracking system to obtain the source multi-object tracking result and the derivative multi-object tracking result; based on the relationship between the source multi-object tracking result and the derivative multi-object tracking result, a robustness verification result of the multi-object tracking system is obtained. By using this method, the robustness of the multi-object tracking system can be tested by verifying whether the source multi-object tracking result and the derivative multi-object tracking result satisfy the metamorphic relationship, avoiding the need for a large amount of manual marking of the expected output data frames in the traditional test method and improving the test efficiency of the robustness of the multi-object tracking system.

[0084] In one embodiment, step S1 includes the following steps:

[0085] S101. When the application scenario of the multi-object tracking system is a scenario of simulating the acceleration of a tracked object, obtain a preset source video test case;

[0086] S102. Extract one picture frame every K frames from the source video test case containing multiple picture frames to obtain a derivative video test case corresponding to the source video test case;

[0087] For a given source video test case V = {f1, f2,..., f n}, extract the picture frames therein at a fixed interval K to obtain a derivative video test case V' = {f'1, f'2,..., f' m}. The mapping relationship between the source video test case and the derivative video test case is:

[0088] f' i = f ik , i ∈ N+, ik ≤ n.

[0089] For example, extract frames at a fixed interval K = 2 to simulate the tracked object traveling at twice the speed, and obtain a derivative test case with 50 frames, that is, m = 50. At this time, the mapping relationship between the source test case and the derivative test case is f' i = f 2i .

[0090] In another embodiment, for a given source video V = {f1, f2, …, f n} containing multiple picture frames, a small number (accounting for p%) of picture frames <q1, q2, …, q r > are randomly deleted to obtain a derivative video V′ = {f′1, f′2, …, f′ m}. The mapping relationship between the picture frames of the source video test case and the derivative video test case is:

[0091]

[0092] For example, 1% of the 100 picture frames in the source video test case are randomly deleted, that is, 1 frame. Assuming that the 30th frame is deleted, that is, r = 30, a derivative test case with 99 frames is obtained, that is, m = 99. At this time, the mapping relationship between the source test case and the derivative test case is;

[0093]

[0094] S103. Based on the mapping relationship between the source video test case and the derivative video test case, construct a metamorphic relationship of the multi-target tracking system in the scenario of simulating the acceleration of the tracked object.

[0095] Specifically, the test server determines the mapping relationship obtained in S102 as the metamorphic relationship of the multi-target tracking system in the scenario of simulating the acceleration of the tracked object.

[0096] In other embodiments, the scenario of simulating the object acceleration also includes the situation of missing tracking video frames. In this case, for a given source video V = {f1, f2, …, f n} containing multiple picture frames, a small number (accounting for p%) of picture frames <q1, q2, …, q r > are randomly deleted to obtain a derivative video V′ = {f′1, f′2, …, f′ m}. The mapping relationship between the picture frames of the source video and the derivative video is:

[0097]

[0098] In an application embodiment, step S1 further includes:

[0099] S104. Randomly select a specific number of reference picture frames from the source video test case containing multiple picture frames, and use the previous picture frame of the reference picture frame to overwrite the reference picture frame to form a derivative video test case corresponding to the source video test case;

[0100] From the preset source video test case V = {f1, f2, …, f n}{Randomly select several reference picture frames {q1, q2, …, q from among them} r}}

[0101] Use the previous picture frame of the reference picture frame to overwrite the reference picture frames {q1, q2, …, q} r} to form a derivative video test case V′ = {f′1, f′2, …, f′ corresponding to the source video test case} m}}

[0102] Based on the source video test case V = {f1, f2, …, f} n} and the derivative video test case V′ = {f′1, f′2, …, f′} m}, construct the metamorphic relationship of the multi - target tracking system in the scenario of simulating the acceleration of the tracked object. The metamorphic relationship is as follows:

[0103]

[0104] For example, the test server randomly overwrites 1% of the 100 frames of the source test case, that is, 1 frame. Assume that the overwritten frame is the thirtieth frame, that is, r = 30, to obtain a derivative test case containing 100 frames, that is, m = 100. At this time, the mapping relationship between the source test case and the derivative test case is;

[0105]

[0106] [[ID=HB]] S105. Based on the mapping relationship between the source video test case and the derivative video test case, construct the metamorphic relationship of the multi - target tracking system in the scenario of simulating the acceleration of the tracked object.

[0107] Specifically, the test server determines the mapping relationship obtained in S104 as the metamorphic relationship of the multi - target tracking system in the scenario of simulating the acceleration of the tracked object.

[0108] In an application embodiment, step S1 further includes:

[0109] S106. Randomly select a specific number of reference picture frames from the source video test case containing multiple picture frames, and pixelate the reference picture frames to obtain a derivative video test case.

[0110] For a given source video V = {f1, f2, …, f} containing multiple picture frames n}, randomly pixelate a small number (accounting for p%) of the picture frames <q1, q2, …, q r >, to obtain a derivative video V′ = {f′1, f′2, …, f′} m . The mapping relationship between the picture frames of the source video and the derivative video is:

[0111]

[0112] where pixelate(f i ) represents pixelating f i , and this metamorphic relationship simulates a scenario of sudden signal failure.

[0113] For example, the test server randomly pixelates 1% of the frame images in 100 frame images of the source test case, that is, 1 frame image. Assuming that the thirty - first frame is pixelated, i.e., r = 30, a derived test case containing 100 frames is obtained, i.e., m = 100. At this time, the mapping relationship between the source test case and the derived test case is;

[0114]

[0115] In other application embodiments, for a given source video V = {f1, f2, …, f n} containing multiple picture frames, randomly blur a small number (accounting for p%) of the picture frames <q1, q2, …, q r >, and a derived video V′ = {f′1, f′2, …, f′ m} is obtained. The mapping relationship between the picture frames of the source video and the derived video is:

[0116]

[0117] where blur(f i ) represents blurring f i , and this metamorphic relationship simulates a scenario of sudden camera out - of - focus.

[0118] For example, the test server randomly blurs 1% of the 100 frames in the test case, that is, 1 frame. Assuming that the thirty - first frame is blurred, i.e., r = 30, a derived test case containing 100 frames is obtained, i.e., m = 100. At this time, the mapping relationship between the source test case and the derived test case is;

[0119]

[0120] S107. Based on the mapping relationship between the source video test case and the derived video test case, construct a metamorphic relationship for the multi - target tracking system in the scenario of simulating the acceleration of the tracked object.

[0121] Specifically, the test server determines the mapping relationship obtained in S106 as the metamorphic relationship for the multi - target tracking system in the scenario of simulating the acceleration of the tracked object.

[0122] In a second aspect, based on the same inventive concept as the embodiment of the robust performance test method, the embodiment of the present application provides a robust performance test device, and this device includes:

[0123] A metamorphosis relationship construction module, which is used to construct a metamorphosis relationship of the multi-object tracking system based on the application scenario of the multi-object tracking system; the metamorphosis relationship refers to the corresponding relationship between the source test case input by the multi-object tracking system and the derived test case.

[0124] A derived case acquisition module, which is used to obtain a derived test case corresponding to the source test case based on the metamorphosis relationship.

[0125] A tracking result acquisition module, which is used to input the source test case and the derived test case into the multi-object tracking system to obtain a source multi-object tracking result and a derived multi-object tracking result.

[0126] A robustness verification module, which is used to obtain a robustness verification result of the multi-object tracking system based on the relationship between the source multi-object tracking result and the derived multi-object tracking result.

[0127] In this device, the test server constructs a metamorphosis relationship of the multi-object tracking system based on the application scenario of the multi-object tracking system; the metamorphosis relationship refers to the corresponding relationship between the source test case input by the multi-object tracking system and the derived test case; based on this metamorphosis relationship, a derived test case corresponding to the source test case is obtained; the source test case and the derived test case are input into the multi-object tracking system to obtain a source multi-object tracking result and a derived multi-object tracking result; based on the relationship between the source multi-object tracking result and the derived multi-object tracking result, a robustness verification result of the multi-object tracking system is obtained. By using this method, the robustness of the multi-object tracking system can be tested by verifying whether the metamorphosis relationship is satisfied between the source multi-object tracking result and the derived multi-object tracking result, avoiding the need for a large amount of manual marking of the expected output data frames in the traditional test method and improving the test efficiency of the robustness of the multi-object tracking system.

[0128] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, where the memory stores a computer program that runs on the processor, and when the processor executes the computer program, it implements the robust performance test method mentioned in the first aspect.

[0129] In a fourth aspect, an embodiment of the present application provides a storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the robust performance test method mentioned in the first aspect.

[0130] It should be noted that in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0131] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories.

[0132] The above are only the specific implementation manners of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. A robust performance testing method, characterized in that, The method includes the following steps: Based on the application scenario of the multi-object tracking system, construct the metamorphic relationship of the multi-object tracking system; the metamorphic relationship refers to the corresponding relationship between the source test cases input to the multi-object tracking system and the derived test cases; Input the source test cases and the derived test cases into the multi-object tracking system to obtain the source multi-object tracking result and the derived multi-object tracking result; Based on the relationship between the source multi-object tracking result and the derived multi-object tracking result, obtain the robustness verification result of the multi-object tracking system; The constructing the metamorphic relationship of the multi-object tracking system based on the application scenario of the multi-object tracking system includes the following steps: When the application scenario of the multi-object tracking system is a scenario of simulating the acceleration of a tracked object, obtain a preset source video test case; Extract one picture frame every K frames from the source video test case containing multiple picture frames to obtain a derived video test case corresponding to the source video test case; Or, When the application scenario of the multi-object tracking system is a scenario of simulating a camera failure, obtain a preset source video test case; Randomly select a specific number of reference picture frames from the source video test case containing multiple picture frames, and use the previous picture frame of the reference picture frames to overwrite the reference picture frames to form a derived video test case corresponding to the source video test case; Or, When the application scenario of the multi-object tracking system is a scenario of simulating a camera failure, obtain a preset source video test case; Randomly select a specific number of reference picture frames from the source video test case containing multiple picture frames, and pixelate the reference picture frames to obtain a derived video test case; Based on the mapping relationship between the source video test case and the derived video test case, construct the metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of a tracked object.

2. The robust performance testing method according to claim 1, characterized in that, The constructing the metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of a tracked object based on the mapping relationship between the source video test case and the derived video test case includes the following steps: From a preset source video test case containing multiple picture frames , extract the picture frames therein at a fixed interval K to obtain a derived video test case ; Based on the source video test case and the derived video test case , construct the metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of the tracked object. The metamorphic relationship is as follows: 。 3. The robust performance test method according to claim 1, wherein, The randomly selecting a specific number of reference picture frames from the source video test case containing multiple picture frames and using the previous picture frame of the reference picture frames to overwrite the reference picture frames to form a derived video test case corresponding to the source video test case includes the following steps: Select several reference picture frames randomly from the preset source video test cases ; ; Cover the reference picture frame with the previous picture frame of the reference picture frame Form a derived video test case corresponding to the source video test case ; Based on the source video test case and the derivative video test case , construct the metamorphic relationship of the multi-object tracking system in the scenario of simulating the acceleration of the tracked object, and the metamorphic relationship is as follows: 。 4. The robust performance testing method according to claim 1, wherein The obtaining the robustness verification result of the multi-object tracking system based on the relationship between the source multi-object tracking result and the derived multi-object tracking result further includes the following steps: If the source multi-object tracking result and the derived multi-object tracking result satisfy the metamorphic relationship, it indicates that the robustness of the multi-object tracking system is good.

5. A robust performance testing device, based on the robust performance testing method according to any one of claims 1-4, characterized in that, The device includes: A metamorphic relationship construction module, which is used to construct the metamorphic relationship of the multi-object tracking system based on the application scenario of the multi-object tracking system; the metamorphic relationship refers to the corresponding relationship between the source test cases input to the multi-object tracking system and the derived test cases; A tracking result acquisition module, which is configured to input the source test case and the derived test case into the multi-target tracking system to obtain a source multi-target tracking result and a derived multi-target tracking result; A robustness verification module, which is configured to obtain a robustness verification result of the multi-target tracking system based on the relationship between the source multi-target tracking result and the derived multi-target tracking result.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 4 is implemented.

Images