Target detection algorithm failure cause analysis method, device and equipment

Abstract

The present disclosure relates to a failure cause analysis method, device and equipment of a target detection algorithm, and belongs to the technical field of target detection. The method comprises: training and / or testing the target detection algorithm through a data set, and obtaining failure data of the target detection algorithm in the training and / or testing process, each piece of failure data comprising a failure type and measurement values of a plurality of failure influence factors; performing causal reasoning on the failure process of the target detection algorithm based on the failure data to obtain failure causal relationships of the target detection algorithm under at least one failure type, the failure causal relationships comprising causal relationships between the plurality of failure influence factors; and iteratively calculating failure weight values of the plurality of failure influence factors based on the failure causal relationships corresponding to each failure type in the at least one failure type. The present disclosure can qualitatively and quantitatively analyze the failure causes of the target detection algorithm.

Description

Technical Field

[0001] This disclosure relates to the field of target detection technology, and in particular to a method, apparatus and equipment for analyzing the failure causes of a target detection algorithm. Background Technology

[0002] Object detection is an important research area in computer vision. Its task is to find all objects of interest in an image or video, which includes two sub-tasks: object localization and object classification, i.e., determining the category and location of objects. With the rapid development of machine learning, object detection algorithms trained on large amounts of data can be applied to more and more real-world fields and perform well, such as face recognition, intelligent transportation, and industrial inspection.

[0003] However, object detection algorithms often suffer from failures, failing to accurately locate and classify single or multiple objects in images or videos. For example, they may struggle to detect occluded objects, objects in poorly lit environments, and objects not present in the training data. Algorithm failures can pose significant risks in real-world applications. For instance, in autonomous driving, inaccurate obstacle identification can render vehicles unable to avoid obstacles, endangering human lives; in intelligent medical diagnosis, incorrect object localization can disrupt subsequent operations. Therefore, as application scenarios demand increasingly higher precision and accuracy from object detection algorithms, the issue of algorithm failures must be taken seriously.

[0004] However, since algorithm failure is caused by a variety of factors, it is complex and unexplainable. How to reverse-engineer the cause of algorithm failure with limited information has become a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This invention proposes a method, apparatus, and equipment for analyzing the failure causes of target detection algorithms. It can automatically determine the causal relationship between algorithm failure and potential failure influencing factors, and provide the failure weight value of each failure influencing factor, thereby realizing qualitative and quantitative analysis of the failure causes of target detection algorithms.

[0006] According to a first aspect of the present disclosure, a method for analyzing the failure causes of a target detection algorithm is provided, comprising:

[0007] The target detection algorithm is trained and / or tested using a dataset, and failure data of the target detection algorithm is obtained during the training and / or testing process. Each failure data includes the failure type and the measurement values ​​of multiple failure influencing factors.

[0008] Based on the failure data, causal reasoning is performed on the failure process of the target detection algorithm to obtain the failure causal relationship of the target detection algorithm under at least one failure type. The failure causal relationship includes the causal relationship between the multiple failure influencing factors.

[0009] For each of the at least one failure type, the failure weight values ​​of the plurality of failure influencing factors are iteratively calculated based on the failure causal relationship corresponding to the failure type. The failure weight values ​​of the failure influencing factors are used to indicate the contribution of the failure influencing factors to the generation of the failure type by the target detection algorithm.

[0010] Optionally, the failure causal relationship is represented by a failure causal graph, where the failure type is the result node in the failure causal graph, and the multiple failure influencing factors are the cause nodes in the failure causal graph; the causal relationship between the multiple failure influencing factors, and the causal relationship between the failure influencing factors and the failure type, are the directed edges in the failure causal graph.

[0011] Optionally, the step of performing causal reasoning on the failure process of the target detection algorithm based on the failure data to obtain the failure causal relationship of the target detection algorithm under at least one failure type includes:

[0012] For each failure type in the at least one failure type, the encoder maps the metric values ​​of multiple failure influencing factors in each failure data containing the failure type to states, and constructs a state space through all embedded states;

[0013] The state space is mapped to the action space by a decoder, and an action is selected from the dynamic space in each decision step, with the order of action selection serving as the causal relationship between the multiple failure influencing factors.

[0014] The parameters of the encoder and the decoder are adjusted by maximizing the reward function through a strategy, and the failure causality of the target detection algorithm under the failure type is output through the decoder with adjusted parameters.

[0015] Optionally, the step of iteratively calculating the failure weight values ​​of the plurality of failure influencing factors based on the failure causal relationship corresponding to each of the at least one failure type includes:

[0016] Assign initial equal weight values ​​to each failure influencing factor in the failure causal relationship corresponding to the failure type;

[0017] For each failure influencing factor, all parent variables of the failure influencing factor are obtained from the failure causal relationship corresponding to the failure type, and the weight value of the failure influencing factor is calculated based on the weight values ​​of all parent variables.

[0018] The weight values ​​of each failure influencing factor are calculated iteratively, and the weight values ​​of each failure influencing factor when the weight values ​​converge are taken as the failure weight values ​​of the failure influencing factors under the failure type.

[0019] According to a second aspect of the present disclosure, a failure cause analysis apparatus for a target detection algorithm is provided, comprising:

[0020] The data acquisition module is used to train and / or test the target detection algorithm using a dataset, and to acquire failure data of the target detection algorithm during the training and / or testing process. Each failure data includes the failure type and the measurement values ​​of multiple failure influencing factors.

[0021] The causal reasoning module is used to perform causal reasoning on the failure process of the target detection algorithm based on the failure data, and to obtain the failure causal relationship of the target detection algorithm under at least one failure type. The failure causal relationship includes the causal relationship between the multiple failure influencing factors.

[0022] The weight calculation module is used to iteratively calculate the failure weight values ​​of the plurality of failure influencing factors for each failure type among the at least one failure type, based on the failure causal relationship corresponding to the failure type. The failure weight values ​​of the failure influencing factors are used to indicate the contribution of the failure influencing factors to the target detection algorithm in generating the failure type.

[0023] Optionally, the failure causal relationship is represented by a failure causal graph, where the failure type is the result node in the failure causal graph, and the multiple failure influencing factors are the cause nodes in the failure causal graph; the causal relationship between the multiple failure influencing factors, and the causal relationship between the failure influencing factors and the failure type, are the directed edges in the failure causal graph.

[0024] Optionally, the causal reasoning module is used for:

[0025] For each failure type in the at least one failure type, the encoder maps the metric values ​​of multiple failure influencing factors in each failure data containing the failure type to states, and constructs a state space through all embedded states;

[0026] The state space is mapped to the action space by a decoder, and an action is selected from the dynamic space in each decision step, with the order of action selection serving as the causal relationship between the multiple failure influencing factors.

[0027] The parameters of the encoder and the decoder are adjusted by maximizing the reward function through a strategy, and the failure causality of the target detection algorithm under the failure type is output through the decoder with adjusted parameters.

[0028] Optionally, the weight calculation module is used for:

[0029] Assign initial equal weight values ​​to each failure influencing factor in the failure causal relationship corresponding to the failure type;

[0030] For each failure influencing factor, all parent variables of the failure influencing factor are obtained from the failure causal relationship corresponding to the failure type, and the weight value of the failure influencing factor is calculated based on the weight values ​​of all parent variables.

[0031] The weight values ​​of each failure influencing factor are calculated iteratively, and the weight values ​​of each failure influencing factor when the weight values ​​converge are taken as the failure weight values ​​of the failure influencing factors under the failure type.

[0032] According to a third aspect of the present disclosure, a computer device is provided, the computer device comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the failure cause analysis method of the target detection algorithm provided in the first aspect of the present disclosure.

[0033] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein computer program instructions are stored on the computer-readable storage medium, and the computer program instructions, when executed by a processor, implement the failure cause analysis method of the target detection algorithm provided in the first aspect of the present disclosure.

[0034] The technical solutions provided in this disclosure have at least the following beneficial effects:

[0035] This invention, based on causal reasoning and iterative calculation, enables qualitative and quantitative analysis of the failure causes of target detection algorithms. Specifically, the invention first collects and records failure data of the target detection algorithm according to a pre-configured format; then, based on the failure data, it performs causal reasoning on the failure process of the target detection algorithm to qualitatively analyze the causal relationship between algorithm failure and potential failure influencing factors; subsequently, iterative calculation is performed based on the failure causal relationship corresponding to each failure type to quantitatively analyze the failure weight value of each failure influencing factor; finally, one or more failure influencing factors with the highest failure weight value are selected as the failure cause, realizing automated reverse reasoning of failure causes and improving the accuracy and efficiency of failure cause analysis of target detection algorithms.

[0036] This invention also provides a comprehensive analysis of potential failure factors in target detection algorithms from both internal and external perspectives. Specifically, it considers the design and training process of the target detection algorithm as internal factors and the properties of sample images / videos in the dataset as external factors. By combining internal and external factors, it addresses potential failure factors that could lead to the failure of target detection algorithms and provides a framework for measuring these potential failure factors. This comprehensive analysis of potential failure factors further improves the accuracy of analysis into the causes of target detection algorithm failures.

[0037] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0039] Figure 1 This is a flowchart illustrating a failure cause analysis method for a target detection algorithm according to an exemplary embodiment.

[0040] Figure 2 This is a schematic diagram illustrating the reasoning process of a failure causal relationship according to an exemplary embodiment.

[0041] Figure 3 This is a schematic diagram illustrating the calculation of failure weight values ​​according to an exemplary embodiment.

[0042] Figure 4 This is a schematic diagram of a failure cause-effect graph according to an exemplary embodiment.

[0043] Figure 5 This is a block diagram of a failure cause analysis apparatus for a target detection algorithm according to an exemplary embodiment.

[0044] Figure 6 This is a block diagram illustrating a computer device 600 according to an exemplary embodiment. Detailed Implementation

[0045] The exemplary embodiments will now be described in detail with reference to the accompanying drawings.

[0046] like Figure 1 As shown, the failure cause analysis method in this invention includes the following steps (steps 1 to 3).

[0047] Step 1: Train and / or test the object detection algorithm using the dataset, and obtain failure data of the object detection algorithm during the training and / or testing process.

[0048] Object detection algorithm failure refers to the algorithm's inability to perform the object detection task as expected (e.g., object localization failure) or the failure to achieve the expected results (e.g., inaccurate object classification). Given a pre-trained or trained object detection algorithm, selecting a suitable dataset and collecting failure data based on that dataset is a preparatory step for failure analysis. The dataset selection can be matched to the application scenario and / or task of the object detection algorithm. For example, if the object detection algorithm is applied to autonomous driving, a dataset of sample images / videos representing various scenarios such as cities, sandy areas, rural areas, plains, and hills can be selected. If the object detection algorithm is applied to medical diagnosis, a dataset of sample images / videos representing various types of medical images can be selected.

[0049] This invention trains and / or tests an object detection algorithm using a dataset. Specifically, sample images / videos from the dataset are input into the algorithm to perform object detection. When the algorithm fails, relevant information about the failure is collected, such as the algorithm's running status, training parameters, and image features that failed detection, forming a failure data record. It's important to note that failure data can be collected during both the training and testing phases of the object detection algorithm. The datasets used for training and testing can be the same or different; this invention does not impose any limitations on this. Each failure data record includes the failure type and the measurement values ​​of multiple failure influencing factors. Therefore, before collecting failure data, it is necessary to design which failure influencing factors to be collected and how each factor will be measured.

[0050] In one example, this invention considers the properties of sample images / videos in the dataset as external factors and the design and training process of the object detection algorithm as internal factors, taking into account the potential failure factors that could lead to the failure of the object detection algorithm from the perspective of combining external and internal factors. For example, the potential failure factors of the object detection algorithm, and the metrics for each failure factor, are shown in Table 1 below.

[0051] Table 1. Potential failure factors and their metrics for target detection algorithms.

[0052]

[0053]

[0054] Step 2: Based on the failure data, perform causal reasoning on the failure process of the target detection algorithm to obtain the failure causal relationship of the target detection algorithm under at least one failure type.

[0055] In this invention, failure data can be classified according to failure type, with failure data of the same failure type grouped into one category. Then, for a specific failure type, causal reasoning is performed on the failure process of the target detection algorithm based on all failure data under that failure type, yielding the failure causal relationship of the target detection algorithm under that failure type. This failure causal relationship includes the causal relationships between multiple failure influencing factors.

[0056] Optionally, the causal relationship of failure can be represented by a failure cause-effect graph, where the failure type is the result node and multiple failure influencing factors are the cause nodes. The causal relationships between multiple failure influencing factors, and the causal relationships between failure influencing factors and failure type, are represented as directed edges in the failure cause-effect graph; the direction of the directed edges is the direction from cause to effect. For example, the failure cause-effect graph is defined as G. f = (V, E), where the node set V includes cause nodes and result nodes, and the edge set E is a set of directed edges connecting nodes. If there is a causal relationship between two nodes, they are connected by directed edges. Where G... f Given a directed acyclic graph, if there exists a path from node v... i Pointing to node v j A directed edge represents a node v. i It will affect node v j The result.

[0057] It should be understood that this invention obtains a failure causality relationship for each failure type, such as a failure causality graph. Since it is initially impossible to determine which failure influencing factors are the cause of a certain failure type, multiple failure influencing factors are the same in the failure data recorded for different failure types. However, since the causal relationship between multiple failure influencing factors may be different under different failure types, the failure causality relationship under different failure types is also different. For example, although the number of nodes and the content of nodes are the same in the failure causality graph under different failure types, the connection between nodes in the final failure causality graph may be different.

[0058] This invention does not limit the specific method of causal inference. Optionally, the causal relationship of failure can be inferred through causal inference algorithms based on independent and identically distributed (IOD), time series, or deep learning. For different target detection algorithm failure cause analysis processes, a suitable causal inference algorithm can be selected according to the actual situation (such as cost, performance, etc.). In one example, for each failure type in at least one failure type, taking the reinforcement learning-based CORL algorithm (Causaldiscovery with Ordering-based Reinforcement Learning) as an example, such as... Figure 2As shown, step 2 above includes the following sub-steps (steps 2.1 to 2.3).

[0059] Step 2.1: For each failure type in at least one failure type, the encoder maps the metric values ​​of multiple failure influencing factors in each failure data containing that failure type to states, and constructs a state space through all embedded states.

[0060] In the CORL algorithm, the variable sorting search problem is transformed into a multi-step Markov Decision Process (MDP). In each decision step, a variable (failure influencing factor) is treated as an action, and the order of the selected actions is regarded as the search sort. Finding the optimal sort means finding the optimal failure causal relationship.

[0061] Studies have shown that feedforward neural network models struggle to capture potential causal relationships by directly using failure data as states, while data preprocessed by an encoder helps find better rankings. Therefore, the encoder first measures the values ​​x of each failure influencing factor in each failure data point. i Mapping to state s i In this context, assuming there are n failure influencing factors (n is a positive integer), all embedded states constitute the state space S: = {s1, ..., s2}. n Let the initial state be... By selecting the first action, the complete state space will be

[0062] Step 2.2: Map the state space to the action space using a decoder, and select an action from the dynamic space in each decision step, using the order of action selection as the causal relationship between multiple failure influencing factors.

[0063] make This represents the actual state encountered in the t-th decision step when generating variables and sorting them. A decoder based on an LSTM (Long Short Term Memory) structure is used to process the state space. Mapping to action space A. Each time, the selected failure factors are masked, restricting each failure factor to be selected only once to generate a valid ranking. In each decision step, from action space A := {v1, ..., v...} n Let} select an action (failure factor). The action space contains all failure factors in each decision step, i.e., |A| = n. The state transition process depends on the failure factor selected in the current decision step. If the failure factor selected in the t-th decision step is v i Then the state will transition to the encoder-embedded s iThe corresponding state s i ∈S, that is

[0064] Step 2.3: Adjust the parameters of the encoder and decoder by maximizing the reward function through the strategy, and output the failure causality of the target detection algorithm under this failure type through the decoder with adjusted parameters.

[0065] In ranking-based methods, only failure factors selected in previous decision-making steps can become potential parent variables for the currently selected failure factor. Two reward mechanisms can be implemented: episodic reward and dense reward. The optimization objective is to learn a reward function that maximizes the policy. Policy gradients can be used to optimize the encoder and decoder, and different policy gradients can be used for contextual rewards and dense rewards. After parameter tuning and optimization, all failure data of a certain failure type are input into the encoder again, and the decoder can output the failure causality relationship of the target detection algorithm under that failure type.

[0066] Optionally, for contextual rewards, the reward function As shown in the following formula.

[0067]

[0068] in, and These are parameters related to the encoder and decoder, which can be adjusted... and Make Maximize; n is the number of failure influencing factors; m is the number of failure influencing factors v i The number of observations, that is, including failure influencing factors v. i The number of failure data points (i.e., the number of failure data points recorded) is a metric, where m is a positive integer. The failure influencing factor v in the kth failure data. i The metric value, or failure influencing factor v i The k-th observation, where k is a positive integer less than or equal to m; Indicates and The observed values ​​of failure influencing factors associated with the nodes in the data. This indicates the position of node v in the variable sorting result. i Previous node set; θ i These are parameters associated with each likelihood value; It refers to the given and θ i In the case of The conditional probability distribution, which represents... The value in the given and θ i The probability of such a situation can be determined using a linear Gaussian model. In this Gaussian model, N represents a Gaussian distribution. express With θ i The inner product, σ, is used to determine the mean of the Gaussian distribution. 2 It is variance.

[0069] Optionally, for dense rewards, a reward value is calculated at each decision step, where the immediate reward function in the i-th decision step is as shown in the following equation.

[0070]

[0071] Step 3: For each failure type in at least one failure type, iteratively calculate the failure weight values ​​of multiple failure influencing factors based on the failure causal relationship corresponding to the failure type.

[0072] The failure weight value indicates the contribution of failure influencing factors to the failure type generated by the target detection algorithm; that is, the probability that a failure influencing factor will cause the target detection algorithm to generate a corresponding failure type, or the importance of the failure influencing factor under that failure type. In an example, such as... Figure 3 As shown, step 3 above includes the following sub-steps (steps 3.1 to 3.3).

[0073] Step 3.1: Assign initial equal weight values ​​to each failure influencing factor in the failure causal relationship corresponding to the failure type.

[0074] For each failure type, the weight values ​​of each failure influencing factor are first initialized based on the failure causal relationship corresponding to that failure type. When the failure causal relationship is represented by the failure causal graph described above, the same initial weight value is first assigned to each cause node (one cause node corresponds to one failure influencing factor) in the failure causal graph. Typically, as follows... Figure 3 As shown, if there are n cause nodes, then the initial weight value of each cause node is...

[0075] Step 3.2: For each failure influencing factor, obtain all parent variables of the failure influencing factor from the failure causal relationship corresponding to the failure type, and calculate the weight value of the failure influencing factor based on the weight values ​​of all parent variables.

[0076] For each failure type, for each potential failure influencing factor of that failure type, all other failure influencing factors that have a causal relationship with the failure influencing factor and whose result is the failure influencing factor are obtained from the failure causal relationship corresponding to that failure type; that is, all parent variables of the failure influencing factor. When the failure causal relationship is represented by the above failure causal graph, all parent variables of a certain cause node (failure influencing factor) refer to all other cause nodes pointing to that cause node. In this invention, the weight value of a failure influencing factor is determined by the weight values ​​of all parent variables of that failure influencing factor; that is, the weight value of a cause node in the failure causal graph is determined by the weight values ​​of all other cause nodes pointing to that cause node.

[0077] Optionally, for any failure type, FR(N) i ) represents the i-th failure influencing factor N i The weight value, FR(N) i1 ), FR(N i2 )…FR(N it L(N) represents the weight values ​​of the t parent variables of the failure influencing factor. i1 ), L(N i2 )…L(N it Let N and T represent the number of outgoing chains of the t parent variables of the failure influencing factor, where t is a positive integer. Then, the i-th failure influencing factor N i The weight value FR(N) i The calculation is as follows.

[0078]

[0079] Where d is the damping factor, which can usually be set to 0.85.

[0080] Step 3.3: Iteratively calculate the weight value of each failure influencing factor, and take the weight value of each failure influencing factor when the weight values ​​converge as the failure weight value of that failure influencing factor.

[0081] In each round of iterative calculation, the weight value of each failure influencing factor can be calculated according to step 3.2 above. The weight values ​​of each failure influencing factor are then continuously adjusted and updated to achieve iterative calculation until the weight values ​​of all failure influencing factors reach a stable state, i.e., weight value convergence. For each failure type, when the weight values ​​converge in the iterative calculation, the current weight value of each failure influencing factor is taken as the final failure weight value of that failure influencing factor under that failure type.

[0082] Through steps 1 to 3 above, the potential failure influencing factors and their failure weight values ​​of the target detection algorithm under different failure types can be calculated. Then, one or more failure influencing factors with higher failure weight values ​​can be used as failure causes, thus realizing the mining of failure causes of the target detection algorithm under different failure types. The accuracy of failure cause analysis is improved by combining qualitative and quantitative analysis.

[0083] The failure cause analysis method provided by this invention will be described below through an experimental process.

[0084] In this experiment, the PASCAL VOC (Visual Object Classes) dataset was selected as the test dataset, and the YOLOv5x version of YOLO (You Only Look Once) was selected as the object detection algorithm.

[0085] This experiment collected failure data from 4325 sample images of YOLOv5 at VOC2007 and selected some of the failure influencing factors from Table 1 above to construct a failure data list. Specifically, this experiment added a failure data collection module to the YOLOv5 object detection algorithm code to record training parameters and relevant data of the detected failure images, such as the confidence of the detected category, brightness, blur, and target object size in the sample images. Sample images whose detection results did not match the ground truth were regarded as failure cases. Some failure influencing factors were selected from Table 1 above as the content of the failure record, as shown in Table 2 below.

[0086] Table 2: List of Failure Data Records

[0087] index category obj_size pose truncated pic_size brightness resolution blurriness contrast status 1 3 0.0028 0 0 187500 88.1328 23.1391 309.3938 63.0783 0 2 3 0.0028 0 0 187500 88.1328 23.1391 309.3938 63.0783 2 3 5 0.0898 3 0 166500 112.0267 209.2928 2694.4672 527.4674 0 4 5 0.4937 2 0 187500 137.9939 152.0949 1999.6719 376.2490 2 5 5 0.0178 1 0 203000 124.4111 310.2723 4109.5007 615.9375 2 6 5 0.2024 0 0 187500 105.0942 348.7420 3802.4825 614.8238 0 7 7 0.0215 0 0 187500 108.6415 111.2439 1642.0756 302.1965 0 8 7 0.0196 0 0 187500 100.1301 431.4227 5518.7040 816.6815 0 9 7 0.0542 0 0 187500 129.3307 76.3216 1108.4441 218.6542 2 10 9 0.2236 0 1 165000 132.0508 59.1355 403.4185 102.4193 0

[0088] The aforementioned failure data record list is organized by detection object, evaluating the detection results for all detection objects in each sample image, and collecting the results and related data of detection errors. Furthermore, the category number represents the true category number of the detection object in the test dataset; the pose number (0, 1, 2, 3, 4) represents "unknown pose," "front," "back," "left," and "right," respectively; obj_size represents the size of the detection object; truncated represents the degree of truncation of the detection object; pic_size represents the size of the sample image; brightness represents the brightness of the sample image; resolution represents the resolution of the sample image; blurriness represents the blurriness of the sample image; contrast represents the contrast of the sample image; and status represents the failure type.

[0089] Based on the collected failure data, causal reasoning can be performed to obtain the following results: Figure 4 The failure cause-effect diagram is shown. From Figure 4 There are 10 nodes in total. The `status` node represents the failure type, so it has no sub-variables and is a result node. The other nine nodes represent different failure influencing factors and are cause nodes. `obj_size`, `category`, `pose`, and `truncated` are cause nodes based on the characteristics of the detected object, while `brightness`, `resolution`, `blurriness`, `contrast`, and `pic_size` are cause nodes based on image features. Experimental results show that even failure influencing factors at different levels are related. For example, image size affects the size of the detected object in the sample image; smaller sample images can contain smaller detected objects. Smaller detected objects lead to increased blurriness, which increases detection difficulty and thus increases the likelihood of failure. Other causal relationships can be analyzed one by one in the failure cause-effect graph.

[0090] Then iteratively calculate the above Figure 4 The weight values ​​of the nine cause nodes were calculated, resulting in the results shown in Table 3 below. In this experiment, the failure influencing factors were divided into three levels according to their failure weight values. Level 1 factors had a failure weight value greater than 0.1, which were the most likely causes of algorithm failure in this experiment. The failure weight values ​​of the failure influencing factors in Levels 2 and 3 decreased sequentially. It should be understood that the classification criteria shown in Table 3 are not fixed and should be adjusted according to different situations in practical applications, potentially resulting in different numbers of levels. Based on Table 3, in the test results of the YOLOv5 object detection algorithm on the VOC2007 dataset, the three most likely failure causes were blurriness, contrast, and resolution. Of course, this result may vary for different object detection algorithms and datasets.

[0091] Table 3. Failure weight values ​​and ranking results of failure influencing factors.

[0092]

[0093] In summary, this invention, based on causal reasoning and iterative calculation, enables qualitative and quantitative analysis of the failure causes of target detection algorithms. Specifically, this invention first collects and records failure data of the target detection algorithm according to a pre-configured format; then, based on the failure data, it performs causal reasoning on the failure process of the target detection algorithm to qualitatively analyze the causal relationship between algorithm failure and potential failure influencing factors; subsequently, iterative calculation is performed based on the failure causal relationship corresponding to each failure type to quantitatively analyze the failure weight value of each failure influencing factor; finally, one or more failure influencing factors with the highest failure weight value are selected as the failure cause, realizing automated reverse reasoning of failure causes and improving the accuracy and efficiency of failure cause analysis of target detection algorithms.

[0094] This invention also provides a comprehensive analysis of potential failure factors in target detection algorithms from both internal and external perspectives. Specifically, it considers the design and training process of the target detection algorithm as internal factors and the properties of sample images / videos in the dataset as external factors. By combining internal and external factors, it addresses potential failure factors that could lead to the failure of target detection algorithms and provides a framework for measuring these potential failure factors. This comprehensive analysis of potential failure factors further improves the accuracy of analysis into the causes of target detection algorithm failures.

[0095] Exemplary device

[0096] Figure 5 This is an exemplary embodiment illustrating a failure cause analysis device for a target detection algorithm, with reference to... Figure 5 The device includes: a data acquisition module 510, a causal reasoning module 520, and a weight calculation module 530.

[0097] The data acquisition module 510 is used to train and / or test the target detection algorithm through the dataset, and to acquire the failure data of the target detection algorithm during the training and / or testing process. Each failure data includes the failure type and the measurement values ​​of multiple failure influencing factors.

[0098] The causal reasoning module 520 is used to perform causal reasoning on the failure process of the target detection algorithm based on the failure data, and to obtain the failure causal relationship of the target detection algorithm under at least one failure type. The failure causal relationship includes the causal relationship between the multiple failure influencing factors.

[0099] The weight calculation module 530 is used to iteratively calculate the failure weight values ​​of the plurality of failure influencing factors based on the failure causal relationship corresponding to each failure type among the at least one failure type. The failure weight values ​​of the failure influencing factors are used to indicate the contribution of the failure influencing factors to the generation of the failure type by the target detection algorithm.

[0100] In one embodiment of this disclosure, the failure causal relationship is represented by a failure causal graph, where the failure type is the result node in the failure causal graph, and the plurality of failure influencing factors are the cause nodes in the failure causal graph; the causal relationship between the plurality of failure influencing factors, and the causal relationship between the failure influencing factors and the failure type, are the directed edges in the failure causal graph.

[0101] In one embodiment of this disclosure, the causal reasoning module 520 is configured to: for each failure type among the at least one failure type, map the metric values ​​of multiple failure influencing factors in each failure data containing the failure type to states through an encoder, and construct a state space through all embedded states; map the state space to an action space through a decoder, and select an action from the dynamic space in each decision step, using the action selection order as the causal relationship between the multiple failure influencing factors; adjust the parameters of the encoder and the decoder through a policy maximization reward function, and output the failure causal relationship of the target detection algorithm under the failure type through the parameter-adjusted decoder.

[0102] In one embodiment of this disclosure, the weight calculation module 530 is configured to: assign initial equal weight values ​​to each failure influencing factor in the failure causal relationship corresponding to the failure type; for each failure influencing factor, obtain all parent variables of the failure influencing factor from the failure causal relationship corresponding to the failure type, and calculate the weight value of the failure influencing factor based on the weight values ​​of all parent variables; iteratively calculate the weight value of each failure influencing factor, and take the weight value of each failure influencing factor when the weight values ​​converge as the failure weight value of the failure influencing factor under the failure type.

[0103] The exemplary device is an embodiment of the device corresponding to the exemplary method described above. The specific operation of each module can be understood with reference to the description of the method embodiment, and will not be repeated here.

[0104] Exemplary electronic devices

[0105] Figure 6 This is a block diagram illustrating a computer device 600 according to an exemplary embodiment. The computer device 600 may be a terminal, a laptop computer, a desktop computer, a server, a computer cluster, or other types of electronic equipment.

[0106] Reference Figure 6 The computer device 600 may include at least one processor 610 and a memory 620. The processor 610 can execute instructions stored in the memory 620. The processor 610 is communicatively connected to the memory 620 via a data bus. In addition to the memory 620, the processor 610 may also be communicatively connected to an input device 630, an output device 640, and a communication device 650 via a data bus.

[0107] Processor 610 can be any conventional processor. Processors may include central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems on chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0108] The memory 620 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0109] In this embodiment of the present disclosure, the memory 620 stores executable instructions, and the processor 610 can read the executable instructions from the memory 620 and execute the instructions to implement all or part of the steps of the failure cause analysis method of the target detection algorithm in the above exemplary embodiment.

[0110] Exemplary computer-readable storage media

[0111] In addition to the methods and apparatus described above, exemplary embodiments of this disclosure also include a computer program product or a computer-readable storage medium storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to perform all or part of the steps described in the exemplary embodiments above.

[0112] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages, and scripting languages ​​(e.g., Python). The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0113] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media include: static random access memory (SRAM) having one or more electrically connected wires, electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk, or any suitable combination thereof.

[0114] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for analyzing the failure causes of an object detection algorithm, wherein the object detection algorithm is used to locate or classify single or multiple objects in an image or video, characterized in that, The method includes: The target detection algorithm is trained and / or tested using a dataset, and failure data of the target detection algorithm is obtained during the training and / or testing process. Each failure data includes the failure type and the measurement values ​​of multiple failure influencing factors. Based on the failure data, causal reasoning is performed on the failure process of the target detection algorithm to obtain the failure causal relationship of the target detection algorithm under at least one failure type. The failure causal relationship includes the causal relationship between the multiple failure influencing factors. For each failure type among the at least one failure type, the failure weight value of the plurality of failure influencing factors is iteratively calculated based on the failure causal relationship corresponding to the failure type. The failure weight value of the failure influencing factor is used to indicate the contribution of the failure influencing factor to the target detection algorithm in generating the failure type. The step of performing causal reasoning on the failure process of the target detection algorithm based on the failure data to obtain the failure causal relationship of the target detection algorithm under at least one failure type includes: For each failure type in the at least one failure type, the encoder maps the metric values ​​of multiple failure influencing factors in each failure data containing the failure type to states, and constructs a state space through all embedded states; The state space is mapped to the action space by a decoder, and an action is selected from the action space in each decision step, with the order of action selection serving as the causal relationship between the multiple failure influencing factors. The parameters of the encoder and the decoder are adjusted by maximizing the reward function through a strategy, and the failure causality of the target detection algorithm under the failure type is output through the decoder with adjusted parameters. Specifically, the step of iteratively calculating the failure weight values ​​of the plurality of failure influencing factors based on the failure causal relationship corresponding to each failure type among the at least one failure type includes: Assign initial equal weight values ​​to each failure influencing factor in the failure causal relationship corresponding to the failure type; For each failure influencing factor, all parent variables of the failure influencing factor are obtained from the failure causal relationship corresponding to the failure type, and the weight value of the failure influencing factor is calculated based on the weight values ​​of all parent variables. The weight values ​​of each failure influencing factor are calculated iteratively, and the weight values ​​of each failure influencing factor when the weight values ​​converge are taken as the failure weight values ​​of the failure influencing factors under the failure type.

2. The method according to claim 1, characterized in that, The failure causal relationship is represented by a failure causal graph, where the failure type is the result node in the failure causal graph, and the multiple failure influencing factors are the cause nodes in the failure causal graph; the causal relationship between the multiple failure influencing factors, and the causal relationship between the failure influencing factors and the failure type, are the directed edges in the failure causal graph.

3. A device for analyzing the failure causes of a target detection algorithm, wherein the target detection algorithm is used to locate or classify single or multiple objects in an image or video, characterized in that, The device includes: The data acquisition module is used to train and / or test the target detection algorithm through a dataset, and to acquire failure data of the target detection algorithm during the training and / or testing process. Each failure data includes the failure type and the measurement values ​​of multiple failure influencing factors. The causal reasoning module is used to perform causal reasoning on the failure process of the target detection algorithm based on the failure data, and to obtain the failure causal relationship of the target detection algorithm under at least one failure type. The failure causal relationship includes the causal relationship between the multiple failure influencing factors. The weight calculation module is used to iteratively calculate the failure weight values ​​of the plurality of failure influencing factors for each failure type in the at least one failure type, based on the failure causal relationship corresponding to the failure type. The failure weight values ​​of the failure influencing factors are used to indicate the contribution of the failure influencing factors to the target detection algorithm in generating the failure type. The step of performing causal reasoning on the failure process of the target detection algorithm based on the failure data to obtain the failure causal relationship of the target detection algorithm under at least one failure type includes: For each failure type in the at least one failure type, the encoder maps the metric values ​​of multiple failure influencing factors in each failure data containing the failure type to states, and constructs a state space through all embedded states; The state space is mapped to the action space by a decoder, and an action is selected from the action space in each decision step, with the order of action selection serving as the causal relationship between the multiple failure influencing factors. The parameters of the encoder and the decoder are adjusted by maximizing the reward function through a strategy, and the failure causality of the target detection algorithm under the failure type is output through the decoder with adjusted parameters. Specifically, the step of iteratively calculating the failure weight values ​​of the plurality of failure influencing factors based on the failure causal relationship corresponding to each failure type among the at least one failure type includes: Assign initial equal weight values ​​to each failure influencing factor in the failure causal relationship corresponding to the failure type; For each failure influencing factor, all parent variables of the failure influencing factor are obtained from the failure causal relationship corresponding to the failure type, and the weight value of the failure influencing factor is calculated based on the weight values ​​of all parent variables. The weight values ​​of each failure influencing factor are calculated iteratively, and the weight values ​​of each failure influencing factor when the weight values ​​converge are taken as the failure weight values ​​of the failure influencing factors under the failure type.

4. The apparatus according to claim 3, characterized in that, The failure causal relationship is represented by a failure causal graph, where the failure type is the result node in the failure causal graph, and the multiple failure influencing factors are the cause nodes in the failure causal graph; the causal relationship between the multiple failure influencing factors, and the causal relationship between the failure influencing factors and the failure type, are the directed edges in the failure causal graph.

5. A computer device, characterized in that, The computer device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the failure cause analysis method of the target detection algorithm according to any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the failure cause analysis method of the target detection algorithm according to any one of claims 1 to 2.

Images