Complex visual image reconstruction method based on depth encoding and decoding dual model

Abstract

The invention discloses a complex visual image reconstruction method based on a depth encoding and decoding dual model and belongs to the visual scene reconstruction technology field in a biomedical image brain decoding. The method is characterized by firstly, collecting and watching functional magnetic resonance signals under a lot of natural images; and then establishing four network models: 1,a coding model, 2, a decoding model, 3, a natural image discrimination model and 4, a visual area response discrimination model, wherein in the coding model, a convolutional neural network is used tocode the natural images into the voxel signals of a visual area; in the decoding model, the convolutional neural network and a deconvolution neural network are used to decode the voxel signals of thevisual area into the natural images; in the natural image discrimination model, true images and false images are discriminated; and in the a visual area response discrimination model, true signals andfalse signals are discriminated. Through training the four designed models, visual scene images can be recovered from a brain signal. In the invention, for the first time, the problem of direct conversion between a natural scene and the brain signal is solved, and the practical application of a brain-computer interface scene can be realized.

Description

technical field [0001] The method belongs to the technical field of visual scene reconstruction in brain decoding of biomedical images, and specifically relates to the framework construction of a natural image reconstruction model of functional magnetic resonance images. Background technique [0002] In 2008, Miyawaki et al. asked subjects to watch a lot of flashing checkerboard stimulus pictures, and recorded the BOLD signal response of these stimuli in the early visual cortex (V1 / V2 / V3), using multi-voxel pattern analysis (Multi-voxel pattern classification, MVPA) method, established a multi-scale sparse multinomial logistic regression (SMLR) local decoder model, realized for the first time the brain signal visual image reconstruction not limited to the candidate visual stimulus category, and reconstructed simple geometric images and letters Stimuli, this study provides a new way of interpreting the visual perception state of the brain. However, the method of Miyawaki et ...

AI Technical Summary

Problems solved by technology

[0003] Aiming at the deficiencies of the background technology, the present invention solves the problems of low accuracy of the reconstructed image, relatively large noise, and relatively long reconstruction time, and improves and designs a dual model based on deep encoding and decoding on the basis of previous research. complex visual image reconstruction method

Images