Bms detection method of artificial intelligence model

Abstract

A BMS detection method of an artificial intelligence model, brain activity characteristics based on the reaction of brain neurons to an image are collected, and intermediate layer characteristics of an artificial intelligence model obtained after the same image data is input into a neural network under test are collected, bidirectional simulation is performed, that is, artificial intelligence model intermediate layer characteristics are used to simulate brain activity characteristics, and brain activity characteristics are used to simulate neural network characteristics, and the similarity of the results obtained by bidirectional simulation is calculated respectively to realize simulation similarity detection.The similarity of brain neuron activity data characteristics and neural network characteristics is obtained by bidirectional simulation of the brain neuron activity data characteristics and the neural network characteristics.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a brain-model-similarity (BMS) detection method for an artificial intelligence model. BACKGROUND

[0002] Deep artificial neural networks (ANN) are partly derived from discoveries in brain visual neuroscience and are now advanced models in many fields of artificial intelligence. More and more work draws on the mechanisms of brain information processing and the ways of neural coding to construct new neural network model structures. At present, there is a lack of methods for evaluating the brain similarity of artificial intelligence models, and by effectively evaluating the similarity between the brain and the neural network model, how to improve the accuracy and stability of the neural network model can be studied and can be used to study the structure and function of the brain, discover new neuron connection patterns, activity patterns and response patterns, etc. SUMMARY

[0003] The present application proposes a BMS detection method for an artificial intelligence model to solve the problem that the prior art cannot describe the relationship between the characteristics of the artificial intelligence model and the characteristics of the brain and cannot evaluate the similarity between the artificial intelligence model and the brain through simulation. The similarity between the two is obtained by bidirectional simulation of the brain neuron activity data characteristics and the neural network characteristics.

[0004] The present application is implemented by the following technical solutions:

[0005] The present application relates to a BMS detection method for an artificial intelligence model. The brain activity characteristics based on the reaction of brain neurons to images and the intermediate layer characteristics of the artificial intelligence model based on the same image data input into the neural network under test are collected respectively, bidirectional simulation is performed, that is, the brain activity characteristics are simulated by using the intermediate layer characteristics of the artificial intelligence model, and the neural network characteristics are simulated by using the brain activity characteristics, and the similarity of the results obtained by bidirectional simulation is calculated respectively to realize simulation similarity detection.

[0006] The brain activity characteristics refer to the neural activity records (V1, V2,..., V 88 ,I1,I2,...,I 168 ) of 88 V4 neurons and 168 IT neurons when observing 8 types of objects in total 2560 image data of macaques.

[0007] The image data refer to the images used to stimulate the vision of macaques to generate brain activity in the public data set.

[0008] The intermediate layer features of the artificial intelligence model refer to: using data enhancement technology to increase the number of image data, then used for fine-tuning training of the model to be evaluated, so that the neural network model has good effect on image classification, and then input the corresponding image to obtain the intermediate layer features (F1, F2,..., F n ) of the artificial intelligence model.

[0009] The data enhancement technology refers to: each time a new image is obtained by rotating the image clockwise by 10 degrees, a total of 35 rotations, so that the number of images is 35 times the number of original images.

[0010] The disclosed data set adopts, but is not limited to, the data set collected by Majaj, Najib J, et al. in Simple learned weighted sums of inferior temporal neuronal firing rates accurately predict human core object recognition performance ([J]. Journal of Neuroscience 35.39 (2015): 13402-13418.).

[0011] The fine-tuning training refers to: on the basis of the trained artificial intelligence model, input a smaller new data set for training.

[0012] The bidirectional simulation specifically includes:

[0013] 1) input the intermediate layer features x = (F1, F2,..., F n ) of the artificial intelligence model extracted by the to-be-tested artificial intelligence model for each image data.

[0014] 2) divide the neural network intermediate layer features X and brain activity features Y extracted by the to-be-tested artificial intelligence model into training set X train ,Y train and test set X test ,Y test according to the corresponding image, and train the regression model Y = aX + b through the training set.

[0015] 3) use the trained regression model and the intermediate layer features X test of the artificial intelligence model in the test set to obtain the simulated brain activity features Y pred , that is, Y pred = aX test + b.

[0016] 4) using the trained regression model and the brain activity features Y in the test set test obtaining the simulated neural network features X pred = ωY test + δ.

[0017] The result similarity, specifically: Wherein: y i is the i-th data in the brain neuron activity data Y test , is the average value of all data in Y test . y′ i is the i-th data in the simulated brain neuron activity data Y pred by the neural network model features, is the average value of all data in Y pred , x i is the i-th data in the neural network model features X test , is the average value of all data in X test X′ i is the i-th data in the simulated neural network model features X pred by the brain neuron activity data, is the average value of all data in X pred .

[0018] The result similarity, which ranges from [0, 1], the greater the result similarity indicates that the neural network model is more similar to the brain, and vice versa, the smaller the value indicates that the neural network model is less similar to the brain.

[0019] The present application relates to a system for implementing the above method, comprising: a data processing module, a model fine-tuning module, a bidirectional simulation module and a correlation evaluation module, wherein: the data processing module respectively inputs the image in the data set after data augmentation to obtain the intermediate layer features of the to-be-tested artificial intelligence model, and pre-processes the brain activity data to obtain the brain activity features; the model fine-tuning module trains and processes the to-be-tested artificial intelligence model according to the data augmented image data set, and obtains the fine-tuned model result; the bidirectional simulation module respectively simulates the brain activity features by the intermediate layer features of the artificial intelligence model and simulates the intermediate layer features of the artificial intelligence model by the brain activity features, and obtains the simulated brain activity features and the simulated intermediate layer features of the artificial intelligence model; the correlation evaluation module calculates the correlation according to the brain activity features, the simulated brain activity features, the intermediate layer features and the simulated intermediate layer features, and realizes the simulation similarity detection.

[0020] Technical effects​

[0021] The application significantly improves the accuracy of evaluating the similarity between the artificial intelligence model and the brain by simulating the intermediate layer features of the artificial intelligence model and the brain activity features. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The flowchart of the application;

[0023] Figure 2 The evaluation of the common model results of the application;

[0024] Figure 3 The evaluation of the simulation of the visual cortex structure network model results of the application. DETAILED DESCRIPTION

[0025] As Figure 1 shown, the BMS detection method of the artificial intelligence model involved in the embodiment includes:

[0026] Step one, according to the disclosed data set including image inducing brain neural activity and brain neural activity data, the 10 data of the same neuron corresponding to the same image are averaged, and then the neural activity data is unfolded to one dimension to obtain the neural activity features; for the intermediate layer features of the artificial intelligence model to be evaluated, considering that the model may not be able to effectively extract the image features in the data set, the model needs to be retrained using the image data in the data set. Considering that the image data is less and the model size is large, data augmentation is used to increase the number of images, and finally the number of images is 10240. After fine-tuning, the original image is input into the artificial intelligence model to be tested, and the intermediate layer output of the artificial intelligence model is extracted as the model features.

[0027] In the data set, the brain neuron activity data includes the neural activity records of 88 V4 neurons and 168 IT neurons of macaques, and the image data includes 2560 gray-scale images of 8 categories of objects.

[0028] The preprocessing refers to removing irrelevant data according to the details of the data set.

[0029] Step two, bidirectional simulation of data, specifically including:

[0030] 2.1) The neural activity feature data obtained in step one and the corresponding artificial intelligence model feature data are divided, and are divided into training set and test set according to the proportion of 7:3 respectively.

[0031] 2.2) Train the regression model obtained in step 2.1 to fit the neural activity feature training dataset, and input the test set obtained in step 2.1 into the trained regression model to obtain the simulated neural activity feature, thereby realizing the simulation of brain activity features using neural network features.

[0032] 2.3) Train the regression model obtained in step 2.2 to fit the model feature training dataset, and input the neural activity feature test dataset obtained in step 2.1 into the trained regression model to obtain the simulated artificial intelligence model feature, thereby realizing the simulation of neural network features using brain neuron activity data.

[0033] 2.4) Repeat steps 2.2 and 2.3 for 10 times to obtain 10 groups of data, wherein each group of data includes: neural activity feature data test set, simulated neural activity feature data, artificial intelligence model feature data test set, and simulated artificial intelligence model feature data.

[0034] Step three, for each group of data obtained in step two, calculate the correlation r1 and r2 respectively, and obtain the score r by averaging; take the average of r of the 10 groups of data to obtain the final artificial intelligence model brain simulation similarity result.

[0035] Through specific actual experiments, under the specific environment setting of Ubuntu 18.1, 64-bit operating system, development language is python, and PyTorch machine learning library, the regression model starts the method with random parameters, and the experimental data obtained is: the recognition accuracy of each model and the simulation similarity result as shown in Table 1 and Table 2.

[0036] First, evaluate the widely used 7 common models CNN, VGG19, ResNet18, AlexNet, VGG11, SqueezeNet, and DenseNet, and Table 1 lists the recognition accuracy of each common model on the image dataset and the obtained brain similarity BMS score, Figure 2 The relationship curve between the recognition accuracy of different models and BMS is shown.

[0037] Table 1: Evaluation results of common model similarity in experiment one

[0038] Model Recognition accuracy BMS CNN 0.862 0.128 VGG19 0.900 0.186 ResNet18 0.9877 0.193 AlexNet 0.9304 0.2158 VGG11 0.9833 0.2088 SqueezeNet 0.9522 0.1988 DenseNet 0.9947 0.2018

[0039] According to the experimental results, it can be seen that the brain similarity BMS of the model is not determined by the recognition accuracy, and the model structure also has a relationship. In fact, the higher the recognition accuracy of the model is, the more similar to the brain it is not necessarily, and the recognition accuracy of some models has actually exceeded the human brain. The experimental results conclude that the brain similarity BMS of the model is not determined by the recognition accuracy, which is consistent with the fact. The complex network model AlexNet, VGG11, DenseNet has a higher score than the simple network model CNN, because the number of neurons and the connection between neurons are more, and to a certain extent, it is more similar to the brain.

[0040] Next, the simulated visual cortex structure network model is evaluated, and the Inhabited_AlexNet, CSNN and Inhabited_CSNN models are selected. AlexNet is selected as the benchmark, and the un-tuned Inhabited_AlexNet, the tuned Inhabited_AlexNet, the un-tuned CSNN, the tuned CSNN and the Inhabited_CSNN are evaluated respectively, and the results are shown in Table 2, and the accuracy and BMS curve and column chart are shown in Figure 3

[0041] Table 2: Experimental two similarity evaluation results of simulated visual cortex structure network model

[0042]

[0043]

[0044] According to the experimental results, it can be found that the simulated visual cortex structure network model has a higher BMS score than the conventional model, although the recognition accuracy of the simulated visual cortex structure network model decreases, which sidefies the effectiveness of the method. And the BMS of the un-tuned model is lower than that of the tuned model, which shows that the model parameters will also affect the similarity evaluation, which is consistent with the fact that the model parameters will affect the working state of different neurons, thereby affecting the similarity of the model and the brain.

[0045] Compared with the prior art, the method adopts a bidirectional simulation technology, so that the brain simulation similarity result of the model is more accurate.

[0046] The above specific embodiments can be adjusted by those skilled in the art in different ways without departing from the principles and purposes of the present application, and the protection scope of the present application is subject to the claims and is not limited by the above specific embodiments, and each implementation scheme within the scope is subject to the constraints of the present application.​

Claims

1. A method for detecting BMS of an artificial intelligence model, the method comprising: The brain activity characteristics based on the reaction of brain neurons to the image and the intermediate layer characteristics of the artificial intelligence model based on the same image data input into the to-be-tested neural network are collected respectively, and bidirectional simulation is performed, that is, the intermediate layer characteristics of the artificial intelligence model are used to simulate the brain activity characteristics, and the brain activity characteristics are used to simulate the neural network characteristics, and then the similarity of the results obtained by bidirectional simulation is calculated respectively to realize simulation similarity detection. The bidirectional simulation specifically includes: 1) inputting the image data of each of the images into the intermediate layer features of the artificial intelligence model to be tested extracted from the artificial intelligence model ; 2) all neural network intermediate layer features X and brain activity features Y extracted from the artificial intelligence model to be tested are divided into data sets according to corresponding images to obtain a training set and a test set , and a regression model is trained through the training set ; 3) Using the trained regression model and the intermediate layer features of the artificial intelligence model in the test set Obtaining simulated brain activity features That is ; 4) Using the trained regression model and the brain activity features in the test set Obtaining simulated neural network features ; The result similarity, in particular: wherein: , , brain neuron activity data the i-th data in, the average of all data in, , neural network model feature simulated brain neuron activity data the i-th data in, the average of all data in, neural network model feature the i-th data in, the average of all data in , brain neuron activity data simulated neural network model feature the i-th data in, the average of all data in. 2.The BMS detection method of an artificial intelligence model according to claim 1, characterized in that, The brain activity features refer to: using public data sets, specifically including 2560 image data of 8 categories of objects observed by macaques, the neural activity records of 88 V4 neurons and 168 IT neurons ; The image data refers to the images used to stimulate the visual brain activity of macaques in the public data set. 3.The BMS detection method of an artificial intelligence model according to claim 1, characterized in that, The intermediate layer features of the artificial intelligence model refer to: using data enhancement technology to increase the number of image data, then using the image data to fine-tune the model to be evaluated, so that the neural network model has good effect on image classification, and then inputting the corresponding image to obtain the intermediate layer features of the artificial intelligence model . 4.The BMS detection method of an artificial intelligence model according to claim 3, characterized in that, The data enhancement technique refers to rotating the image 10 degrees clockwise each time to obtain a new image, and rotating a total of 35 times, so that the number of images is 35 times the number of original images. 5.The BMS detection method of an artificial intelligence model according to claim 3, characterized in that, The fine-tuning training refers to inputting a smaller new data set into the trained artificial intelligence model for further training. 6.The BMS detection method of the artificial intelligence model according to any one of claims 1-5, characterized in that, specifically It includes: Step one, according to the public data set including images that induce brain neural activity and brain neural activity data, pre-processing is performed, 10 data of the same neuron corresponding to the same image are averaged, and then the neural activity data is unfolded into one dimension to obtain neural activity characteristics; For the intermediate layer characteristics of the artificial intelligence model to be evaluated, the image data in the data set is used to retrain the model, and measures such as rotating the image are used for data enhancement to increase the number of images, and finally the number of images is 10240; after fine-tuning, the original image is input into the to-be-tested artificial intelligence model, and the intermediate layer output of the artificial intelligence model is extracted as the model characteristics; The pre-processing refers to removing irrelevant data according to the details of the data set; Step two, bidirectional simulation data, specifically including: 2.1) The neural activity characteristic data and the corresponding artificial intelligence model characteristic data obtained in step one are divided, and are divided into training set and test set according to the proportion of 7:3 respectively; 2.2) The training set obtained in step 2.1 is used to train the regression model to fit the neural activity characteristic training data set, and after training, the test set obtained in step 2.1 is input into the trained regression model to obtain the simulated neural activity characteristics, realizing the simulation of brain activity characteristics by neural network characteristics; 2.3) The neural activity characteristic training data set obtained in step 2.1 is used to train the regression model trained in step 2.2 to fit the model characteristic training data set, and the neural activity characteristic test data set obtained in step 2.1 is input into the trained regression model to obtain the simulated artificial intelligence model characteristics, realizing the simulation of neural network characteristics by brain neuron activity data; 2.4) Steps 2.2 and 2.3 are repeated 10 times to obtain 10 groups of data, wherein each group of data includes: neural activity characteristic data test set, simulated neural activity characteristic data, artificial intelligence model characteristic data test set and simulated artificial intelligence model characteristic data; Step 3: For each set of data obtained in Step 2, calculate the correlation. and The score is obtained by averaging. ; 10 sets of data The final similarity result of the artificial intelligence model brain simulation is obtained by taking the average.

7. A system for implementing the BMS detection method of the artificial intelligence model according to any one of claims 1-6, characterized in that, It includes: The data processing module, the model fine-tuning module, the bidirectional simulation module and the correlation evaluation module, wherein: the data processing module respectively inputs the images in the data set after data augmentation to obtain intermediate layer features, and pre-processes the brain activity data to obtain brain activity features; the model fine-tuning module trains and processes the to-be-tested artificial intelligence model according to the data augmented image data set to obtain a fine-tuned model result; the bidirectional simulation module respectively simulates the brain activity features from the intermediate layer features of the artificial intelligence model and the intermediate layer features from the brain activity features to obtain simulated brain activity features and simulated intermediate layer features of the artificial intelligence model; and the correlation evaluation module calculates the correlation according to the brain activity features, the simulated brain activity features, the intermediate layer features and the simulated intermediate layer features to realize the simulation similarity detection.

Images