A method for evaluating the generalization ability of abstract reasoning models based on deep networks

Abstract

The present invention belongs to the technical field of machine learning, and specifically relates to a method for evaluating the generalization ability of model abstract reasoning based on a deep network, comprising: obtaining abstract elements and setting preset rules; transforming the abstract elements using the preset rules to obtain a small number of samples; inputting the small number of samples into a deep model to generate a large number of samples; inputting the large number of samples into a trained tested reasoning model for testing to obtain an evaluation result of the model abstract reasoning generalization ability; the present invention can automatically and flexibly construct an abstract reasoning data set required for evaluating the abstract reasoning ability of a deep network according to the preset rules, and can construct a large number of new samples based on the original abstract elements from a small number of samples through the basic elements in the samples as a test set to evaluate the generalization ability of the deep network, thereby improving the efficiency of evaluating the generalization ability of the model abstract reasoning.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a method for evaluating the abstract reasoning generalization ability of a model based on a deep network. Background Art

[0002] In recent years, neural networks have been increasingly used in various fields, especially in the fields of image and speech recognition, where great results have been achieved compared with previous related technologies. Machines imitate human visual, auditory and thinking activities through deep learning algorithms, making great progress in the field of AI and solving many complex problems in pattern recognition. At the same time, neural network models have achieved good results in related fields such as search technology, language processing, computer vision, and recommendation technology. Deep learning, as a hot field in machine learning, aims to build and imitate neural networks that learn and analyze by imitating human brain activities. It imitates the mechanism of the human brain to interpret data by learning specific samples and then training models.

[0003] At present, the training and testing of generalization ability in the field of deep learning mainly improves the generalization ability of the model by data augmentation of labeled samples in order to expand the training samples. The evaluation of generalization ability is to test the trained model through the test set of original data and use evaluation indicators to evaluate its generalization ability. However, the model may only learn statistical data superficially, rather than learning the deep abstract reasoning of labeled samples. At the same time, since the training set and the test set are from similar distribution information, it is not very convincing to test the generalization performance of the model with the test set in the original data set. Although deep models have achieved many results recently, countless experiments have also proved that models with good generalization ability in their own experimental samples can only solve one or several types of problems. Although these models have high accuracy and recognition rate, it is difficult to perform abstract concept reasoning. Summary of the invention

[0004] In order to solve the problems existing in the above-mentioned prior art, the present invention proposes a method for evaluating the abstract reasoning generalization ability of a model based on a deep network, the method comprising: obtaining abstract elements and setting preset rules; transforming the abstract elements using the preset rules to obtain a small number of samples; inputting the small number of samples into the deep model to generate a large number of samples; inputting the large number of samples into the trained tested reasoning model for testing to obtain the evaluation results of the model's abstract reasoning generalization ability; screening the model according to the evaluation results of the model's abstract reasoning generalization ability to obtain the optimal tested reasoning model.

[0005] Preferably, the abstract elements are various abstract elements constructed through known sample data, wherein the abstract elements are constructed based on the original data set in the source domain. The data set uses the public data set NEU-DET of steel defects collected by Northeastern University. A small number of images of each defect are selected, and threshold segmentation is performed on them to obtain a set of background and target, and the target set is used as the abstract element.

[0006] Preferably, setting the preset rules includes: setting a four-tuple (t, o, m, r), taking the changing pattern represented by the four-tuple data as the preset rule, and the four elements of the four-tuple are respectively represented as attribute t, object o, measure m and relationship r; wherein the attributes include shape size, pixel value, color and position, the objects include element type, line and shape, the measures include rotation, reduction, splicing and fusion, and the relationships include XOR, decrement and intersection.

[0007] Preferably, the process of transforming abstract elements using preset rules includes: converting the coordinates of two abstract elements into homogeneous coordinates; splicing the homogeneous coordinates of the two elements into a matrix, constructing a position matrix representing the coordinates of the vertices of the new sample shape in the abstract element based on the spliced ​​matrix, solving the position matrix to obtain the transformation matrix T; multiplying the transformation matrix T by the initial matrix to obtain the new sample shape matrix; using the transformation matrix T to process the homogeneous coordinates of the pixel points of the abstract elements to obtain the position coordinates of the pixel points in the new sample shape; and gathering all the abstract elements processed by the preset rules to obtain a small number of samples.

[0008] Preferably, the process of using a deep model to process a small number of samples includes: the deep model is a diffusion model; a small number of samples are input into the diffusion model, and then subjected to forward sampling of a Markov chain, that is, Gaussian noise is added to the input samples through multiple accumulations during the diffusion process to obtain samples with a standard Gaussian noise distribution; the samples with the standard Gaussian noise distribution are input into the diffusion model for inverse diffusion training, and in the inverse diffusion, the probability distribution of the Markov chain in the inverse diffusion is obtained by maximum likelihood estimation; the distribution of the input samples is restored according to the probability distribution of the Markov chain to generate new samples and pseudo labels.

[0009] Preferably, the process of training the inference model under test includes:

[0010] S1: Obtain a data set in the source domain and preprocess the data set; the preprocessing includes cleaning and normalizing the data in the data set, dividing the normalized data, and obtaining a training data set;

[0011] S2: Build the inference model under test and initialize the model;

[0012] S3: Input the data in the training set into the inference model under test for forward propagation to obtain the model output results and model parameters;

[0013] S4: Calculate the loss function of the model based on the model output results and model parameters;

[0014] S5: Input the training data into the inference model under test for back propagation, and use the gradient descent algorithm to optimize the model parameters;

[0015] S6: Repeat steps S3 to S5. When the loss function converges, the model training is completed.

[0016] S7: Input a large number of samples into the trained inference model to evaluate its generalization ability.

[0017] Beneficial effects of the present invention:

[0018] The present invention can automatically and flexibly construct an abstract reasoning data set required for evaluating the abstract reasoning ability of a deep network according to preset rules, and can construct a large number of new samples based on the original abstract elements from a small number of samples through the basic elements in the samples as a test set to evaluate the generalization ability of the deep network, thereby improving the efficiency of evaluating the generalization ability of the model's abstract reasoning. In addition, the evaluation results of the model's generalization performance can better distinguish whether the model has only learned the statistical information of the data or deep-level abstract reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the method for evaluating the generalization ability of model abstract reasoning based on deep network of the present invention;

[0020] Figure 2 A flowchart for generating a large number of samples for the deep generator model of the present invention;

[0021] Figure 3 The present invention is a flowchart of the inference model under test processing a large number of samples. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] A method for evaluating the generalization ability of abstract reasoning models based on deep networks, such as Figure 1As shown, the method includes: obtaining abstract elements and setting preset rules; using the preset rules to transform the abstract elements to obtain a small number of samples; inputting the small number of samples into the deep model to generate a large number of samples; inputting the large number of samples into the trained tested reasoning model for testing to obtain the evaluation results of the model's abstract reasoning generalization ability; screening the model according to the evaluation results of the model's abstract reasoning generalization ability to obtain the optimal tested reasoning model.

[0024] A specific implementation method of a method for evaluating the generalization ability of model abstract reasoning based on a deep network mainly includes a tested reasoning model, a deep generator model, and training samples. The tested reasoning model learns the association relationship between abstract elements based on the labeled training samples. The system also includes several preset rules in different forms. The preset rules are used to automatically create a data set with a regularity of change between basic elements. However, there are difficulties in creating a large number of samples. The deep generator model generates a large number of samples according to the preset rules for testing the tested reasoning model.

[0025] Tested reasoning model: The tested reasoning model can be a model in many fields, such as classic deep networks such as CNN and ResNet. In the present invention, the target detection model and WReN in the field of computer vision are used as examples. The tested reasoning model is used to complete specified tasks, such as classification, regression, etc. The tested reasoning model first learns the characteristics of the data set through the original data set input, such as DeepMind uses neural networks to complete the visual IQ test (RPMs). Through the input questions and answers, the deep network will learn the characteristics of the input data and some correlations between elements. Its main role in the present invention is to evaluate the generalization ability of the tested reasoning model by automatically generating data sets.

[0026] Training samples: Training samples refer to the training set of the original data set. They are the samples that must first be input into the inference model under test to train the model. In RPMs, they contain 8 pictures and 8 multiple choices. In target detection, they are a variety of different types of targets and manually calibrated boxes, such as cats, dogs, and people.

[0027] Furthermore, the inference model under test learns the association relationship between abstract elements based on the training samples. After the training samples are input into the model, the model feature extraction layer will extract image features such as shape, size, and color. By learning the association relationship between abstract elements in the training samples, such as when learning the incremental relationship, eight pictures are input, and the elements in each of these eight pictures will have an extra triangle, the model will learn to count the number of graphics and other attributes in each of these eight pictures, find its incremental relationship, and then classify them according to the eight multiple-choice pictures input to get the score of each option, and the highest score is the answer. It is specially noted that this training process has only one relational constraint, that is, the increase in the number of shapes, and there is no relational constraint on other attributes such as color change.

[0028] It should be noted that the training samples, the tested inference models and the method of training the models in the above steps are not limited.

[0029] Figure 2 is a flow chart of S2 of the present invention, as Figure 2 As shown in S203, constructing abstract elements, setting preset rules to generate a small number of samples, and inputting the small number of samples into the deep generator model to generate a large number of samples, the method includes but is not limited to the following steps:

[0030] After completing the training of the above-mentioned deep model for the original data set, further, abstract elements can be extracted or constructed by yourself in the training samples. You can choose to crop the target object or directly use the original data, or you can choose to manually construct some simple abstract elements such as triangles, squares, etc. There is no limitation on the construction method of the abstract elements and the form and content of the abstract elements.

[0031] Set preset rules. The preset rules can be set according to different fields. For example, in RPMs, a four-tuple (t, o, m, r) can be set to represent the change rule. The four elements of the four-tuple represent attributes (t), objects (o), measures (m), and relationships (r). Attributes can include shape size, geometric type, pixel value, color, number of targets, target position, etc. Objects can include target types, lines, geometric shapes, etc. Measures can include rotation, reduction, etc. Relationships can include XOR, decreasing, AND, intersection, increasing, etc. The above rules can be combined to form a variety of change rules, so that the newly generated samples have more association combinations different from the original data based on the original data. For example, in the increasing relationship described above, only the number of images increases. At this time, according to the increase in the number of images in the original data set, the color of the image is changed or the triangle in the above is changed into a square or a circle. Thus, one association relationship is converted into two or more association relationships, so that the newly generated samples have more diverse relationships than the original samples. First-order logic formulas can also be used to model diverse and irregular changes, and a matrix automatic generator is constructed to achieve the purpose of testing the reasoning ability of model logic and geometric evolution.

[0032] The process of transforming abstract elements using preset rules includes: converting the coordinates of two abstract elements into homogeneous coordinates; splicing the homogeneous coordinates of the two elements into a matrix, constructing a position matrix representing the coordinates of the vertices of the new sample shape in the abstract element based on the spliced ​​matrix, solving the position matrix to obtain the transformation matrix T; multiplying the transformation matrix T by the initial matrix to obtain the new sample shape matrix; using the transformation matrix T to process the homogeneous coordinates of the pixel points of the abstract elements to obtain the position coordinates of the pixel points in the new sample shape; and collecting all the abstract elements processed by the preset rules to obtain a small number of samples.

[0033] In particular, the preset rules are not limited to this. For example, in the target detection algorithm, the preset rules can be changed to operations such as splicing, overlapping, and changing the target background color. Taking the defect detection algorithm as an example, deep models such as yolo or fasterrcnn are trained on the original data set. After the model classifies and regresses the labeled samples, it uses the preset rules to flip some defect samples of the original defect data, overlap multiple defects, change the defect background color, or randomly crop defects, so as to generate a new data set to evaluate the generalization performance of the model.

[0034] The process of using a deep model to process a small number of samples includes: the deep model is a diffusion model; a small number of samples are input into the diffusion model, and then subjected to a forward sampling of a Markov chain, that is, Gaussian noise is added to the input samples through multiple accumulations during the diffusion process to obtain samples with a standard Gaussian noise distribution; the samples with a standard Gaussian noise distribution are input into the diffusion model for inverse diffusion training, and in inverse diffusion, the probability distribution of the Markov chain in inverse diffusion is obtained through maximum likelihood estimation; the distribution of the input samples is restored according to the probability distribution of the Markov chain to generate new samples.

[0035] Furthermore, two methods are used to automatically generate datasets.

[0036] The first method: After obtaining the abstract elements and presetting the rules as described above, select a small number of abstract elements to input into the deep generator model. Before each input of the abstract element batchsize, that is, before an iteration, use the preset rules to change the abstract elements to generate a small number of samples, and then input the small number of samples into the deep generator model. The deep generator model can select GAN, VAE, Diffusion Model and other models. Finally, the deep generative network generates a large number of samples.

[0037] The second method is to construct abstract elements, which include various abstract elements in known sample data or manually constructed. You can choose to manually construct elements such as squares and triangles, or crop and extract target objects in training samples, set the preset rules as described above, input the abstract elements into the guided diffusion model, and introduce the preset rule y into the guided diffusion model. In addition to inputting random Gaussian noise into the diffusion model, the rule y is also added to the noise estimation model. Combined with conditional and unconditional training methods, the model back-propagates to simulate the predicted gradient direction, control the sampling and generation of the reverse process, and change the mean center of its reverse propagation, so that the diffusion model can be generated unconditionally and conditionally.

[0038] The main formula is as follows:

[0039]

[0040] Among them, ∈ θ (x t ) represents the noise estimation model, ∈ θ (x t ,y) represents adding rule y to the noise estimation model; s is a coefficient, and its size will affect the controllability and authenticity of the generation. Conditional prediction minus unconditional prediction can replace the guidance of gradient by the diffusion model guided by the classifier.

[0041] Diffusion model: There are some noise input models that obey the standard Gaussian distribution, which transform the input data into pure Gaussian noise, and then restore the noise after forward propagation back to the input data. This model is very similar to the GAN network, both of which generate data through noise, but the noise and data of the diffusion model are of the same dimension.

[0042] Forward diffusion formula:

[0043]

[0044] Among them, x t is the picture at the tth moment; is the cumulative multiplication of the weight term at the tth moment, which is known prior knowledge and controls the added noise; t is the noise added at time t.

[0045] The forward propagation process is the process of adding noise that follows a standard Gaussian distribution to the data. This is the key to understanding the diffusion model and building training samples. The added noise will serve as a label for model training. The formula for obtaining the mean through reverse propagation is as follows:

[0046]

[0047] Among them, z t is the noise to be estimated at each moment; α t is the weight term at the tth moment, which is known prior knowledge and controls the added noise; β t is the attenuation function at the tth moment, is a known term, α t =1-β t .

[0048] The reverse process is the denoising and restoration process of the model. By gradually obtaining the reversed distribution, the model can restore the input data from a completely standard Gaussian distribution. By embedding preset rules into the diffusion model, the model is trained through forward and backward propagation, and finally generates a large amount of data as a test set.

[0049] Figure 3 is the flow chart of S4 of the present invention, as Figure 3As shown, the method includes but is not limited to the following steps: after obtaining a large amount of data through a deep generative model, use it as a test set to test the inference model under test, and obtain the test effect of the inference model under test on the automatically generated data set by using evaluation indicators such as recall rate, average precision, and accuracy, and then evaluate the generalization performance of the inference model under test, and can better judge whether the model has learned abstract reasoning.

[0050] The process of training the inference model under test includes:

[0051] S1: Obtain a data set in the source domain and preprocess the data set; the preprocessing includes cleaning and normalizing the data in the data set, dividing the normalized data, and obtaining a training data set;

[0052] S2: Build the inference model under test and initialize the model;

[0053] S3: Input the data in the training set into the inference model under test for forward propagation to obtain the model output results and model parameters;

[0054] S4: Calculate the loss function of the model based on the model output results and model parameters;

[0055] S5: Input the training data into the inference model under test for back propagation, and use the gradient descent algorithm to optimize the model parameters;

[0056] S6: Repeat steps S3 to S5. When the loss function converges, the model training is completed.

[0057] S7: Input a large number of samples into the trained inference model to evaluate its generalization ability.

[0058] In this embodiment, the tested reasoning model consists of two parts: a backbone module and a neck module. The backbone module has a 9-layer structure, including 5 convolutional layers, 4 C2F modules and an SPPF pooling module. The backbone module is used to extract features, and the neck module is a feature pyramid fusion network, the purpose of which is to fuse feature information and enhance the location and semantic information of the features. At the end of the neck part is the detection layer, and three feature layers are output in the feature fusion network for target detection.

[0059] In this embodiment, the inference model under test is composed of a combination of multiple Stages; the first layer of Stage0 is a convolution layer with a convolution kernel size of 7×7, followed by a BN layer, and then the ReLU function is used for nonlinear activation, and the second layer is a maximum pooling layer; Stage1 and subsequent Stages all contain two Bottleneck structures; Bottleneck1 is three convolution layers plus the ReLU function, and a residual branch with a convolution kernel of 1×1 is connected in parallel; Bottleneck2 is three convolution layers plus the ReLU function, and Bottleneck2 is directly residually connected to the input without affecting the number of its output channels.

[0060] The process of training the inference model under test includes: taking a large number of samples generated in the target domain as input samples, inputting the samples into the inference model under test for prediction; extracting shallow features of the input samples by the shallow network of the inference model under test; extracting deep features of the input samples by the deep network; fusing the shallow features with the deep features to obtain fused features; obtaining prediction results based on the fused features; comparing the prediction results with the true labels, and evaluating the results using the accuracy, precision, and recall performance indicators.

[0061] The loss function of the model is expressed as follows: The loss function uses the softmax cross entropy loss function. Its expression is:

[0062]

[0063] Where N is the number of samples, C is the number of categories, and y ij represents the label of sample i, represents the predicted probability of sample i in category j.

[0064] Adam is used as the optimizer. Adam is a commonly used adaptive optimization algorithm. It adaptively adjusts the learning rate according to the first-order moment estimate and the second-order moment estimate of the gradient, thereby optimizing the parameters in the neural network. The process of parameter optimization is as follows: first, the gradient calculation is performed, and the gradient of the softmax function with respect to the parameters is calculated during the training process. Then, the variables are initialized, and the first-order moment estimate and the second-order moment estimate of the gradient are initialized, usually initialized to 0. Then, the first-order moment estimate and the second-order moment estimate are updated respectively by the exponentially weighted moving average. Subsequently, in order to eliminate the deviation of the first-order moment estimate and the second-order moment estimate during initialization, they are divided by the corresponding deviation correction coefficient. Finally, the parameters are updated according to the values ​​of the first-order moment estimate and the second-order moment estimate.

[0065] Specifically, the process of optimizing the model parameters using the gradient descent algorithm includes: the loss functions used by the tested inference model are classification loss and regression loss, the classification loss is VFL Loss, and the regression loss is in the form of CIOU Loss+DFL; the specific process includes: initial model variables, calculating the gradient of the model loss function with respect to the parameters; initializing the first-order moment estimate and the second-order moment estimate of the gradient, and the expression is:

[0066] m t =β 1 ·m t-1 +(1-β 1 )·g t

[0067] v t =β 2 ·v t-1 +(1-β 2 )·(g t ) 2

[0068] Among them, m t and v t Respectively represent the first-order and second-order moment estimates of the gradient at time t, β 1 and β 2 represents the smoothing parameter, g t is the gradient at time t;

[0069] The first-order moment estimate and the second-order moment estimate are updated respectively by exponentially weighted moving average, and the expressions are as follows:

[0070]

[0071] Among them, θ t represents the parameter at time t, η represents the learning rate, ε is a constant that is not 0, and is m t and v t The deviation correction is expressed as:

[0072]

[0073]

[0074] The updated first-order moment estimate and second-order moment estimate are divided by their corresponding bias correction coefficients to obtain optimized first-order moment estimate and second-order moment estimate; the model parameters are updated according to the optimized first-order moment estimate and second-order moment estimate.

[0075] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the generalization ability of model abstract reasoning based on deep network, characterized in that: include: Obtain abstract elements and set preset rules; Abstract elements are various abstract elements constructed by known sample data, where the abstract elements are constructed based on the original data set in the source domain. The data set uses the steel defect public data set NEU-DET. A small number of defect images of each type are selected, and threshold segmentation is performed on them to obtain a set of background and target. The target set is used as the abstract element; the abstract elements are transformed using preset rules to obtain a small number of samples; a small number of samples are input into the deep model to generate a large number of samples; Input a large number of samples into the trained inference model to be tested, and obtain the evaluation results of the model's abstract reasoning and generalization ability; The models are screened according to the evaluation results of the model's abstract reasoning and generalization capabilities to obtain the optimal tested reasoning model; Setting the preset rule includes: setting a four-tuple (t, o, m, r), taking the change law represented by the four-tuple data as the preset rule, and the four elements of the four-tuple are respectively represented as attribute t, object o, measure m and relationship r; wherein the attributes include shape size, pixel value, color and position, the objects include element type, line and shape, the measures include rotation, reduction, splicing and fusion, and the relationships include XOR, decrement and intersection; The process of transforming abstract elements using preset rules includes: converting the coordinates of two abstract elements into homogeneous coordinates; splicing the homogeneous coordinates of the two elements into a matrix, constructing a position matrix representing the coordinates of the vertices of the new sample shape in the abstract element based on the spliced ​​matrix, solving the position matrix to obtain the transformation matrix T; multiplying the transformation matrix T by the initial matrix to obtain the new sample shape matrix; using the transformation matrix T to process the homogeneous coordinates of the pixel points of the abstract elements to obtain the position coordinates of the pixel points in the new sample shape; and collecting all the abstract elements processed by the preset rules to obtain a small number of samples.

2. According to claim 1, a method for evaluating the generalization ability of model abstract reasoning based on deep network is characterized in that: The process of using a deep model to process a small number of samples includes: the deep model is a diffusion model; a small number of samples are input into the diffusion model, and then subjected to a forward sampling of a Markov chain, that is, Gaussian noise is added to the input samples through multiple accumulations during the diffusion process to obtain samples with a standard Gaussian noise distribution; the samples with a standard Gaussian noise distribution are input into the diffusion model for inverse diffusion training, and in inverse diffusion, the probability distribution of the Markov chain in inverse diffusion is obtained through maximum likelihood estimation; the distribution of the input samples is restored according to the probability distribution of the Markov chain to generate new samples and pseudo labels.

3. According to claim 1, a method for evaluating the generalization ability of model abstract reasoning based on deep network is characterized in that: The process of training the inference model under test includes: S1: Obtain a data set in the source domain and preprocess the data set; the preprocessing includes cleaning and normalizing the data in the data set, dividing the normalized data, and obtaining a training data set; S2: Build the inference model under test and initialize the model; S3: Input the data in the training set into the inference model under test for forward propagation to obtain the model output results and model parameters; S4: Calculate the loss function of the model based on the model output results and model parameters; S5: Input the training data into the inference model under test for back propagation, and use the gradient descent algorithm to optimize the model parameters; S6: Repeat steps S3 to S5. When the loss function converges, the model training is completed. S7: Input a large number of samples into the trained inference model to evaluate its generalization ability.

4. According to claim 3, a method for evaluating the generalization ability of model abstract reasoning based on deep network is characterized in that: The tested inference model consists of two parts: the backbone module and the neck module. The backbone module has a 9-layer structure, including 5 convolutional layers, 4 C2F modules and an SPPF pooling module. The backbone module is used to extract features, and the neck module is a feature pyramid fusion network, which aims to fuse feature information and enhance the location and semantic information of features. At the end of the neck part is the detection layer, which outputs three feature layers in the feature fusion network for target detection.

5. According to claim 4, a method for evaluating the generalization ability of model abstract reasoning based on deep network is characterized in that: The process of training the inference model under test includes: taking a large number of samples generated in the target domain as input samples, inputting the samples into the inference model under test for prediction; extracting shallow features of the input samples by the shallow network of the inference model under test; extracting deep features of the input samples by the deep network; fusing the shallow features with the deep features to obtain fused features; obtaining prediction results based on the fused features; comparing the prediction results with the true labels, and evaluating the results using the accuracy, precision, and recall performance indicators.

6. According to claim 3, a method for evaluating the generalization ability of model abstract reasoning based on deep network is characterized in that: The process of optimizing the model parameters using the gradient descent algorithm includes: the loss functions used by the tested inference model are classification loss and regression loss, the classification loss is VFL Loss, and the regression loss is in the form of CIOU Loss+DFL; the specific process includes: initial model variables, calculating the gradient of the model loss function with respect to the parameters; initializing the first-order moment estimate and the second-order moment estimate of the gradient, and the expression is: m t =β1·m t-1 +(1-β1)·g t v t =β2·v t-1 +(1-β2)·(g t ) 2 Among them, m t and v t They represent the first-order and second-order moment estimates of the gradient at time t, β1 and β2 represent the smoothing parameters, and g t is the gradient at time t; The first-order moment estimate and the second-order moment estimate are updated respectively by exponentially weighted moving average, and the expressions are as follows: Among them, θ t represents the parameter at time t, η represents the learning rate, ε is a constant that is not 0, and is m t and v t The deviation correction is expressed as: The updated first-order moment estimate and second-order moment estimate are divided by their corresponding bias correction coefficients to obtain optimized first-order moment estimate and second-order moment estimate; the model parameters are updated according to the optimized first-order moment estimate and second-order moment estimate.

Images