CNN interpretation graph object correlation analysis method based on correlation analysis

Abstract

The invention relates to a CNN interpretation graph object correlation analysis method based on correlation analysis, and belongs to the technical field of image processing. And by introducing and combining object correlation analysis, contribution to each object in the interpretation graph and quantitative analysis and calculation of the correlation degree between the objects are realized. According to the method, the false correlation can be effectively discriminated, the established false correlation can be accurately positioned, and the method provides a scientific basis for identifying and eliminating the false correlation, so that the interpretation capability of the CNN model is enhanced, and the method has wide applicability and can be effectively applied to various scenes regardless of the model structure.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a CNN interpretation graph object correlation analysis method based on correlation analysis. Background Art

[0002] Convolutional Neural Networks (CNNs) have achieved great success in image and video based tasks such as image classification, object detection, semantic segmentation, etc. However, due to their inherent unexplainable behavior, the application and deployment of CNN-based systems in safety-critical real-world applications such as healthcare, automation, and assistive robotics are limited. These industries require reliable and explainable systems that require trust in the decision-making process with no or very low error tolerance. In addition, when a system fails, its decision-making process should be defendable and explainable to avoid its recurrence in the future. Therefore, it is challenging to trust a black box system without understanding the reasoning process behind its predictions.

[0003] According to the theory of stable learning, the failure of OOD generalization is attributed to false associations. For example, the model mistakenly identifies the field as a dog's feature and the snow as a wolf's feature. Once the background of the dog and the wolf changes, the performance of the model drops sharply. Therefore, a trustworthy model must be able to identify and eliminate false correlations.

[0004] The research on image object correlation of CNN classification model can be divided into three categories, namely: verification analysis of correlation existence, extraction of classification-related objects, and establishment of object correlation.

[0005] 1. Correlation existence verification analysis aims to verify whether the model can identify correlations in images. The main methods are: filter visualization, by visualizing the filters of the convolutional layer and observing their responses on different images; sensitivity analysis, by making small perturbations to the input image and observing the changes in the model output.

[0006] 2. Classification-related object extraction, the purpose is to explain the image area that the model focuses on during the classification process. The main methods are: block occlusion, by occluding different parts of the image to observe the changes in the model output; random input sampling explanation, by randomly changing a part of the input image to observe its impact on the classification result; class activation mapping (CAM), by back-propagation to calculate the contribution of each pixel to the classification result; reverse gradient calculation, by calculating the gradient of the input image to generate an explanation heat map.

[0007] 3. Establishing object correlation relationships, the purpose is to transition from pixel-level interpretation to concept-level interpretation and establish correlation relationships between objects. The main methods are: decision tree, which explains CNN classification decisions through decision tree models; knowledge graph interpretation, which explains the relationship between objects in the image by building a knowledge graph. Summary of the invention

[0008] The purpose of the present invention is to provide a CNN interpretation graph object correlation analysis method based on correlation analysis to address the deficiencies of object correlation analysis in the existing CNN image classification model interpretation graph, introduce combined object correlation analysis, and realize quantitative analysis and calculation of the contribution of each object in the interpretation graph and the correlation between objects. This method can effectively identify false correlations and accurately locate established false correlations. This method provides a scientific basis for identifying and eliminating false correlations, thereby enhancing the interpretation ability of the CNN model, and has wide applicability. Regardless of the model structure, it can be effectively applied to various scenarios.

[0009] To achieve the above objectives, the technical solution of the present invention is: a CNN explanation graph object correlation analysis method based on correlation analysis, which introduces combined object correlation analysis to achieve quantitative analysis and calculation of the contribution of each object in the explanation graph and the correlation between objects.

[0010] In one embodiment of the present invention, the method comprises the following steps:

[0011] Step 1: Foreground and background separation: First, separate the foreground and background of image X to obtain the mask M b ;

[0012] Step 2: Superpixel segmentation: Apply the obtained mask to the image and perform superpixel segmentation on the image with only the foreground, that is, the image with only the foreground X M =M b *X, to X M Perform superpixel segmentation and obtain a superpixel set A = {A1, A2, A3, ..., A n}, A i Corresponding to the i-th pixel block, 1≤i≤n, i and n are natural numbers, and n is the number of pixel blocks after segmentation; for each pixel block A j The pixel coordinate set contained in is expressed as k j is the kth pixel in the corresponding jth pixel block;

[0013] Step 3: Randomly collect analysis samples: randomly extract pixel blocks in A to form a random superpixel block set M. M={…,A j,…}, j is a natural number, 1≤j≤n, and each superpixel block A obtained in step 2 j and the corresponding pixel coordinates P j , super pixel block set M i The corresponding pixel coordinate set O i ={…,P j , …}, i is a natural number; create an image X0 with the same size as the original image X and all pixel values ​​are 0, and superimpose all superpixel blocks in M ​​on X0 according to the coordinate position; for those that do not belong to M i The pixel point, that is Randomly extract pixels in According to the coordinate position, it is superimposed on X0 to obtain a random sampling sample N i ; For N i , construct a one-dimensional vector R according to the sampling of the pixel block i =[r1,r2,…,r n ,],when When j =1, otherwise r j =0; N i Input into CNN model f to get the N i The model classification value f(N i ), thus obtaining the complete information R corresponding to a sample i =[r1,r2,…,r n ,f(N i )]; L random superpixel sets are repeatedly generated by random sampling to form a set {M1,M2,…,M L}, and obtain L random samples to form a set {N1,N2,…,N L} and the set {R1,R1,…,R n};

[0014] Step 4: Linear regression model fitting: The n pixel blocks of the segmentation result are regarded as n features [X1,X1,…,X n ], generating a linear model containing multiple independent variables; the linear regression model is expressed as follows:

[0015]

[0016] Among them, Y is the classification of the randomly sampled sample, β0 is the intercept term, β i is the independent variable X i The regression coefficient, γ j is the interaction term X j1 *X j2 The regression coefficient, ∈ is the error term, n is the number of pixel blocks, q is the number of interaction terms between the two, The least squares method is used to estimate the regression coefficients in the linear regression model to minimize the sum of squared errors between the model predictions and the actual observed values; the least squares method is implemented as follows:

[0017]

[0018] Among them, Y i is the ith observation value, is the i-th value predicted by the model, and L is the number of random sampling samples;

[0019] Step 5. Correlation Analysis: Calculate the Total Sum of Squares Regression sum of squares and residual sum of squares in, is the observed value Y i The mean of

[0020] Step 6: Implement relevant explanation: explain CNN and analyze the correlation between each pair;

[0021] Step 7: Build a knowledge graph based on the analysis results to visualize the relevant analysis.

[0022] In one embodiment of the present invention, step five specifically includes the following sub-steps:

[0023] Sub-step 1: For each feature and interaction term in the linear regression model constructed by formula (1), construct a set D = […, X i ,…,…,(X j1 *X j2 ),…], and calculate the contribution value for each item. i , remove X from the set D i , expressed as D\X i , reconstruct the model based on formula (1), re-estimate the regression coefficient based on formula (2); calculate the regression sum of squares for the model Get X i The sum of squares

[0024] In one embodiment of the present invention, step five further includes the following sub-steps:

[0025] Sub-step 2: Calculate the mean square value: X i The mean square value of X i The degrees of freedom are due to the sample construction in step 3. Calculate the residual mean square: Residual degrees of freedom DF Res=Lnq-1, L is the number of random sampling samples, n is the number of features, and q is the number of interaction terms between the two.

[0026] In one embodiment of the present invention, step five further includes the following sub-steps:

[0027] Sub-step 3: Implementation of F test: By calculating the ratio of the mean square error of each regression coefficient to the residual mean square error, an F test is implemented to determine whether the influence of each variable or interaction term on the target variable is statistically significant;

[0028] The calculation formula of F is as follows:

[0029]

[0030] In one embodiment of the present invention, step five further includes the following sub-steps:

[0031] Sub-step 4: p-value calculation: The p-value indicates that if the H0 hypothesis is true, that is, if X i The probability of observing an F value more extreme than the current F value has no significant effect on the results. The calculation formula is as follows:

[0032]

[0033] Under the null hypothesis, the F value in the F distribution is greater than or equal to probability.

[0034] In one embodiment of the present invention, in step six, CNN is interpreted, and the specific method for analyzing the correlation between each pair is: the correlation is determined according to the p-value, and the correlation is further quantified according to the F-value to obtain the main effects of all feature items and the interaction effects of the interaction items between each pair.

[0035] The present invention also provides a CNN interpretation graph object correlation analysis system based on correlation analysis, comprising:

[0036] Foreground and background separation module;

[0037] Superpixel segmentation module;

[0038] Random sampling module;

[0039] Linear regression model fitting module;

[0040] Correlation analysis module;

[0041] Implement relevant explanation modules;

[0042] Knowledge graph building module;

[0043] Among them, the foreground and background separation module, superpixel segmentation module, random sampling module, linear regression model fitting module, correlation analysis module, implementation of relevant interpretation module, and knowledge graph construction module respectively execute the method steps of step 1 to step 7 described above.

[0044] The present invention also provides a computer software system, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, any of the method steps described above can be implemented.

[0045] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0046] Compared with the prior art, the present invention has the following beneficial effects: the present invention has important value in increasing the stability of the model and the specific advantages are as follows:

[0047] 1. The interpretation method of the present invention is easier to understand;

[0048] 2. The method adopted by the present invention improves the speed and efficiency of analysis and increases the stability of analysis.

[0049] 3. The present invention effectively solves the key problems in the prior art and provides an efficient, stable and accurate CNN correlation analysis solution, laying a foundation for the explanatory research and practical application of CNN models. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure is a flow chart of the method of the present invention.

[0051] Figure 2 is a background segmentation map of an image X according to an embodiment of the present invention.

[0052] Figure 3 A superpixel segmentation map according to an embodiment of the present invention.

[0053] Figure 4 is a superpixel segmentation set according to an embodiment of the present invention.

[0054] Figure 5 FIG. 4 is a mask diagram according to an embodiment of the present invention.

[0055] Figure 6 FIG. 4 is a schematic diagram of random sampling according to an embodiment of the present invention.

[0056] Figure 7 Explanation graph and relevance knowledge graph obtained for the model classification of clothing.

[0057] Figure 8 Explanation graph and relevance knowledge graph obtained for the model classification of clothing.

[0058] Fig. 9 Explanation graph and relevance knowledge graph obtained for the model classification of clothing. DETAILED DESCRIPTION

[0059] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0060] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0061] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0062] like Figure 1-9 As shown, this embodiment provides a CNN interpretation graph object correlation analysis method based on correlation analysis, and the specific implementation steps are as follows:

[0063] Step 1: Foreground and background separation: First, separate the foreground and background of image X to obtain the mask M b ;

[0064] Step 2: Superpixel segmentation: Apply the obtained mask to the image and perform superpixel segmentation on the image with only the foreground, that is, the image with only the foreground X M =M b *X, to X M Perform superpixel segmentation and obtain a superpixel set A = {A1, A2, A3, ..., A n}, A i Corresponding to the i-th pixel block, 1≤i≤n, i and n are natural numbers, and n is the number of pixel blocks after segmentation; for each pixel block A j The pixel coordinate set contained in is expressed as k j is the kth pixel in the corresponding jth pixel block;

[0065] Step 3: Randomly collect analysis samples: randomly extract pixel blocks in A to form a random superpixel block set M. M={…,A j,…}, j is a natural number, 1≤j≤n, and each superpixel block A obtained in step 2 j and the corresponding pixel coordinates P j , super pixel block set M i The corresponding pixel coordinate set O i ={…,P j , …}, i is a natural number; create an image X0 with the same size as the original image X and all pixel values ​​are 0, and superimpose all superpixel blocks in M ​​on X0 according to the coordinate position; for those that do not belong to M i The pixel point, that is Randomly extract pixels in According to the coordinate position, it is superimposed on X0 to obtain a random sampling sample N i ; For N i , construct a one-dimensional vector R according to the sampling of the pixel block i =[r1,r2,…,r n ,],when When j =1, otherwise r j =0; N i Input into CNN model f to get the N i The model classification value f(N i ), thus obtaining the complete information R corresponding to a sample i =[r1,r2,…,r n ,f(N i )]; L random superpixel sets are repeatedly generated by random sampling to form a set {M1,M2,…,M L}, and obtain L random samples to form a set {N1,N2,…,N L} and the set {R1,R1,…,R n};

[0066] Step 4: Linear regression model fitting: The n pixel blocks of the segmentation result are regarded as n features [X1,X1,…,X n ], generating a linear model containing multiple independent variables; the linear regression model is expressed as follows:

[0067]

[0068] Among them, Y is the classification of the randomly sampled sample, β0 is the intercept term, β i is the independent variable X i The regression coefficient, γ j is the interaction term X j1 *X j2 The regression coefficient, ∈ is the error term, n is the number of pixel blocks, q is the number of interaction terms between the two, The least squares method is used to estimate the regression coefficients in the linear regression model to minimize the sum of squared errors between the model predictions and the actual observed values; the least squares method is implemented as follows:

[0069]

[0070] Among them, Y i is the ith observation value, is the i-th value predicted by the model, and L is the number of random sampling samples;

[0071] Step 5. Correlation Analysis: Calculate the Total Sum of Squares Regression sum of squares and residual sum of squares in, is the observed value Y i The mean of

[0072] Sub-step 1: For each feature and interaction term in the linear regression model constructed by formula (1), construct a set D = […, X i ,…,…,(X j1 *X j2 ),…], and calculate the contribution value for each item. i , remove X from the set D i , expressed as D\X i , reconstruct the model based on formula (1), re-estimate the regression coefficient based on formula (2); calculate the regression sum of squares for the model Get X i The sum of squares

[0073] Sub-step 2: Calculate the mean square value: X i The mean square value of X i The degrees of freedom are due to the sample construction in step 3. Calculate the residual mean square: Residual degrees of freedom DF Res =Lnq-1, L is the number of random sampling samples, n is the number of features, and q is the number of interaction terms between the two.

[0074] Sub-step 3: Implementation of F test: By calculating the ratio of the mean square error of each regression coefficient to the residual mean square error, an F test is implemented to determine whether the influence of each variable or interaction term on the target variable is statistically significant;

[0075] The calculation formula of F is as follows:

[0076]

[0077] Sub-step 4: p-value calculation: The p-value indicates that if the H0 hypothesis is true, that is, if X i The probability of observing an F value more extreme than the current F value has no significant effect on the results. The calculation formula is as follows:

[0078]

[0079] Under the null hypothesis, the F value in the F distribution is greater than or equal to probability.

[0080] Step 6: Implement relevant interpretation: interpret CNN, analyze the correlation between each pair, determine the correlation based on the p-value, further quantify the correlation based on the F-value, and obtain the main effects of all feature items and the interaction effects of the interaction items between each pair.

[0081] Step 7: Build a knowledge graph based on the analysis results to visualize the relevant analysis.

[0082] In this example, the parameters are selected as follows: the number of superpixels is 5, 8, 12, and 15, the sampling samples are 100, 200, 400, and 800, and the duplicate samples are removed. The preset significance level is 0.01.

[0083] This example uses the widely used resnet50 pre-training, which can distinguish 1000 categories.

[0084] The test samples in this example are from the validation sets of ILSVRC 2012 and ILSVRC 2015. The test system for this example is win11, the graphics card is GeForce RTX 4080Laptop GPU, and the test environment for this example is PyCharm Professional Edition 2023.3, Python 3.9.13, torch 2.1.2, numpy 1.24.1, pandas 2.2.0 and OpenCV 4.9.0.

[0085] The present invention also provides a CNN interpretation graph object correlation analysis system based on correlation analysis, comprising:

[0086] Foreground and background separation module;

[0087] Superpixel segmentation module;

[0088] Random sampling module;

[0089] Linear regression model fitting module;

[0090] Correlation analysis module;

[0091] Implement relevant explanation modules;

[0092] Knowledge graph building module;

[0093] Among them, the foreground and background separation module, superpixel segmentation module, random sampling module, linear regression model fitting module, correlation analysis module, implementation of relevant interpretation module, and knowledge graph construction module respectively execute the method steps of step 1 to step 7 described above.

[0094] The present invention also provides a computer software system, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, any of the method steps described above can be implemented.

[0095] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0096] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0097] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0100] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A CNN interpretation graph object correlation analysis method based on correlation analysis, characterized in that: The object correlation analysis is introduced to achieve quantitative analysis and calculation of the contribution of each object in the explanation graph and the correlation between objects.

2. According to the method for analyzing the correlation of CNN interpretation graph objects based on correlation analysis in claim 1, it is characterized in that: The steps include: Step 1: Foreground and background separation: First, separate the foreground and background of image X to obtain the mask M b ; Step 2: Superpixel segmentation: Apply the obtained mask to the image and perform superpixel segmentation on the image with only the foreground, that is, the image with only the foreground X M =M b *X, to X M Perform superpixel segmentation and obtain a superpixel set A = {A1, A2, A3, ..., A n }, A i Corresponding to the i-th pixel block, 1≤i≤n, i and n are natural numbers, and n is the number of pixel blocks after segmentation; for each pixel block A j The pixel coordinate set contained in is expressed as k j is the kth pixel in the corresponding jth pixel block; Step 3: Randomly collect analysis samples: randomly extract pixel blocks in A to form a random superpixel block set M. j is a natural number, 1≤j≤n, and each superpixel block A obtained in step 2 j and the corresponding pixel coordinates P j , superpixel block set M i The corresponding pixel coordinate set O i ={…,P j , …}, i is a natural number; create an image X0 with the same size as the original image X and all pixel values ​​are 0, and superimpose all superpixel blocks in M ​​on X0 according to the coordinate position; for those that do not belong to M i The pixel point, that is Randomly extract pixels in According to the coordinate position, it is superimposed on X0 to obtain a random sampling sample N i ; For N i , construct a one-dimensional vector R according to the sampling of the pixel block i =[r1,r2,…,r n ,],when When j =1, otherwise r j =0; N i Input into CNN model f to get the N i The model classification value f(N i ), thus obtaining the complete information R corresponding to a sample i =[r1,r2,…,r n ,f(N i )]; L random superpixel sets are repeatedly generated by random sampling to form a set {M1,M2,…,M L }, and obtain L random samples to form a set {N1,N2,…,N L } and the set {R1,R1,…,R n }; Step 4: Linear regression model fitting: The n pixel blocks of the segmentation result are regarded as n features [X1,X1,…,X n ], generating a linear model containing multiple independent variables; the linear regression model is expressed as follows: Among them, Y is the classification of the randomly sampled sample, β0 is the intercept term, β i is the independent variable X i The regression coefficient, γ j is the interaction term X j1 *X j2 The regression coefficient, ∈ is the error term, n is the number of pixel blocks, q is the number of interaction terms between the two, The least squares method is used to estimate the regression coefficients in the linear regression model to minimize the sum of squared errors between the model predictions and the actual observed values; the least squares method is implemented as follows: Among them, Y i is the ith observation value, is the i-th value predicted by the model, and L is the number of random sampling samples; Step 5. Correlation Analysis: Calculate the Total Sum of Squares Regression sum of squares and residual sum of squares in, is the observed value Y i The mean of Step 6: Implement relevant explanation: explain CNN and analyze the correlation between each pair; Step 7: Build a knowledge graph based on the analysis results to visualize the relevant analysis.

3. The method for analyzing the correlation of CNN explanation graph objects based on correlation analysis according to claim 2, characterized in that: Step 5 specifically includes the following sub-steps: Sub-step 1: For each feature and interaction term in the linear regression model constructed by formula (1), construct a set D = […, X i ,…,…,(X j1 *X j2 ),…], and calculate the contribution value for each item. i , remove X from the set D i , expressed as D\X i , reconstruct the model based on formula (1), re-estimate the regression coefficient based on formula (2); calculate the regression sum of squares for the model Get X i The sum of squares 4. The method for analyzing the object correlation of a CNN explanation graph based on correlation analysis according to claim 3, characterized in that: Step 5 also includes the following sub-steps: Sub-step 2: Calculate the mean square value: X i The mean square value of X i The degrees of freedom are due to the sample construction in step 3. Calculate the residual mean square: Residual degrees of freedom DF Res =Lnq-1, L is the number of random sampling samples, n is the number of features, and q is the number of interaction terms between the two.

5. The method for analyzing the correlation of CNN explanation graph objects based on correlation analysis according to claim 4, characterized in that: Step 5 also includes the following sub-steps: Sub-step 3: Implementation of F test: By calculating the ratio of the mean square error of each regression coefficient to the residual mean square error, an F test is implemented to determine whether the influence of each variable or interaction term on the target variable is statistically significant; The calculation formula of F is as follows:

6. The method for analyzing the correlation of CNN explanation graph objects based on correlation analysis according to claim 5, characterized in that: Step 5 also includes the following sub-steps: Sub-step 4: p-value calculation: The p-value indicates that if the H0 hypothesis is true, that is, if X i The probability of observing an F value more extreme than the current F value has no significant effect on the results. The calculation formula is as follows: Under the null hypothesis, the F value in the F distribution is greater than or equal to F Xi probability.

7. The method for analyzing the correlation of CNN explanation graph objects based on correlation analysis according to claim 6, characterized in that: In step six, CNN is interpreted and the specific method for analyzing the correlation between each pair is as follows: the correlation is determined based on the p-value, and the correlation is further quantified based on the F-value to obtain the main effects of all feature items and the interaction effects of the interaction items between each pair.

8. A CNN interpretation graph object correlation analysis system based on correlation analysis, characterized in that: include: Foreground and background separation module; Superpixel segmentation module; Random sampling module; Linear regression model fitting module; Correlation analysis module; Implement relevant explanation modules; Knowledge graph building module; Among them, the foreground and background separation module, the superpixel segmentation module, the random sampling module, the linear regression model fitting module, the correlation analysis module, the implementation of the correlation interpretation module, and the knowledge graph construction module respectively execute the method steps of step 1 to step 7 as described in any one of claims 2-7.

9. A computer software system, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method steps described in any one of claims 1 to 7 can be implemented.

10. A computer-readable storage medium storing computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 7 can be implemented.

Images