Deep learning based post-stroke aphasia patient fMRI localization method

Abstract

The present application relates to the technical field of brain function network analysis, and particularly relates to a post-stroke aphasia patient fMRI focal recognition method based on deep learning. The present application obtains a region consistency value by matching the geometric structure and the gray scale distribution similarity of the local region of two adjacent fMRI images in an fMRI image sequence, and divides the same time window by using the region consistency value. The present application obtains a functional connection coefficient by the difference of the region consistency value, the geometric structure feature and the gray scale distribution feature of the local region of the fMRI images in two adjacent time windows, and determines the convolution kernel size of the fMRI images in the time window for deep learning, and performs focal recognition on the fMRI images in the time window. The present application analyzes the functional connection coefficient of the coordination degree of the brain region activity mode in the adjacent time window, and adaptively determines the convolution kernel size of the time window by using the functional connection coefficient, thereby improving the recognition effect of the local lesion of the patient.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of brain function network analysis, and particularly relates to a post-stroke aphasia patient fMRI focal recognition method based on deep learning. BACKGROUND

[0002] Post-stroke aphasia is one of the common sequelae of stroke patients, which seriously affects the communication ability and quality of life of patients. Functional magnetic resonance imaging (fMRI) is widely used in brain function research. fMRI technology can capture the dynamic activity of different brain regions in the resting state and when performing specific language tasks, thereby revealing the brain region network related to language function. Deep learning, as a powerful data-driven method, can automatically extract features through the construction of deep neural networks, thereby effectively identifying and classifying complex brain function activity patterns; not only can it improve the recognition accuracy of damaged brain regions, but also can reveal the neural mechanisms of different language function disorders.

[0003] Convolutional Neural Networks (CNN) is a deep learning model specifically designed for image processing and computer vision tasks. The core feature of the CNN model is to use convolutional layers and pooling layers to extract features from images, and these features are used for classification or regression tasks through fully connected layers. Traditional CNN models extract spatial features through two-dimensional or three-dimensional convolution kernels, and convolution operations only act on a single time point or a short time segment, lacking the ability to capture long-range temporal dependencies. However, fMRI dynamic functional connectivity requires analysis of the change pattern of functional connectivity matrix across time windows. When CNN models are used for fMRI dynamic functional connectivity analysis, the short-term nature may lead to misjudgment of brain network dynamic reorganization, resulting in poor recognition of local lesions in post-stroke aphasia patients' fMRI. SUMMARY

[0004] To solve the technical problem of CNN model short-term nature causing errors in fMRI dynamic functional connectivity analysis, leading to poor recognition of local lesions in post-stroke aphasia patients' fMRI, the purpose of the present application is to provide a post-stroke aphasia patient fMRI focal recognition method based on deep learning, and the technical solution adopted is as follows:

[0005] The present application provides a post-stroke aphasia patient fMRI focal recognition method based on deep learning, which comprises:

[0006] Obtaining the fMRI image sequence of the brain of a post-stroke aphasia patient, wherein the fMRI images in the sequence contain different local regions;

[0007] The local regions of each two adjacent fMRI images in the sequence are matched, the region consistency value of the corresponding two fMRI images is obtained according to the geometric structure similarity and the gray distribution similarity of the two matched local regions of each two adjacent fMRI images, and the image matching degree, and the sequence corresponding period is divided into different time windows based on the region consistency value;

[0008] According to the difference of the region consistency value of the fMRI images in each two adjacent time windows, and the difference of the geometric structure features and the gray distribution features of the local regions, a functional connection coefficient of the corresponding two time windows is obtained.

[0009] According to the functional connection coefficient, the convolution kernel size for deep learning of the fMRI images in each time window is determined, and focal recognition is performed on the fMRI images in each time window.

[0010] Further, the region consistency value of the corresponding two fMRI images is obtained, including:

[0011] A matching region pair is formed by the two matched local regions of each two adjacent fMRI images, and a region overall difference value of the corresponding two fMRI images is obtained according to the difference of the gray value of the pixel points of the two local regions in all matching region pairs of each two adjacent fMRI images and the slope difference between adjacent pixel points on the edge.

[0012] According to the number of region matching pairs of each two adjacent fMRI images and the region overall difference value, a region consistency value of the two fMRI images is obtained.

[0013] Further, the region overall difference value of the corresponding two fMRI images is obtained, including:

[0014] The slopes between all adjacent two pixel points on the edge of the local region are sequentially arranged to obtain a slope sequence, and the mean value of the gray values of the pixel points in the local region is calculated as an overall gray value.

[0015] The product of the absolute value of the difference of the overall gray values of the two local regions in each matching region pair of each two adjacent fMRI images and the DTW value of the slope sequence is calculated, and the mean value of all products is taken as the region overall difference value of the corresponding two fMRI images.

[0016] The calculation formula of the region overall difference value of each two adjacent fMRI images is as follows:

[0017] ; The region overall difference value of the cth and (c+1)th fMRI images in the fMRI image sequence is the number of matched region pairs of the cth and (c+1)th fMRI images in the fMRI image sequence; the overall gray value of the local region in the cth fMRI image in the mth matched region pair of the cth and (c+1)th fMRI images in the fMRI image sequence; the overall gray value of the local region in the (c+1)th fMRI image in the mth matched region pair of the cth and (c+1)th fMRI images in the fMRI image sequence; the DTW value of the slope sequence of the two local regions in the mth matched region pair of the cth and (c+1)th fMRI images in the fMRI image sequence; the absolute value function.

[0018] Further, the sequence corresponding time period is divided into different time windows based on the region consistency value, comprising:

[0019] The fMRI images in the fMRI image sequence are clustered based on the region consistency value, to obtain different clustering clusters;

[0020] Each fMRI image in the fMRI image sequence corresponds to a time point; whether the fMRI images in each clustering cluster are continuous in the fMRI image sequence is determined, if yes, the time points corresponding to the fMRI images with the smallest and largest subscripts in each clustering cluster are taken as the starting time point and the ending time point of a time window in turn;

[0021] If not, the fMRI images in each clustering cluster are divided into different continuous image segments based on the continuity of the fMRI images in the fMRI image sequence, and the time points corresponding to the fMRI images with the smallest and largest subscripts in each continuous image segment are taken as the starting time point and the ending time point of a time window in turn.

[0022] Further, the functional connection coefficient corresponding to two time windows is obtained, comprising:

[0023] According to the geometric structure feature and the gray distribution feature of the local region of the fMRI images in each time window, the image feature value of the corresponding time window is obtained;

[0024] The mean value of the region consistency value of all fMRI images in each time window is calculated, and the product of the absolute value difference of the mean value of each adjacent two time windows and the absolute value difference of the image feature value is negatively correlated and normalized to obtain the functional connection coefficient corresponding to two time windows;

[0025] The calculation formula of the functional connection coefficient of each adjacent two time windows is as follows:

[0026] In the formula, the functional connection coefficient of each adjacent two time windows is calculated as follows: is the functional connection coefficient of the d, d+1th time window; is the mean value of the region consistency value of all adjacent two fMRI images in the dth time window; is the mean value of the region consistency value of all adjacent two fMRI images in the d+1th time window; is the image feature value of the dth time window; is the image feature value of the d+1th time window; exp is the exponential function with the natural constant as the base; is the absolute value function.

[0027] Further, the image feature value of the corresponding time window is obtained, comprising:

[0028] For each fMRI image, the absolute value of the difference between the overall gray value of each local region of the fMRI image and the mean value of the gray value of the pixel points in the fMRI image, and the ratio of the variance of the gray value of the pixel points in each local region, are calculated, and the cumulative sum of all the ratios is taken as the gray feature value;

[0029] The absolute value of the difference between the mean value of the elements in the slope sequence of each local region of the fMRI image and the mean value of the elements in the slope sequence of all local regions, and the ratio of the variance of the elements in the slope sequence of each local region, are calculated, and the cumulative sum of all the ratios is taken as the structure feature value;

[0030] The mean value of the product of the gray feature value and the structure feature value of all fMRI images in each time window is taken as the image feature value of each time window; the calculation formula of the image feature value is as follows:

[0031] , , ; In the formula, K is the image feature value of each time window; V is the total number of fMRI images in each time window; is the gray feature value of the vth fMRI image in each time window; is the structure feature value of the vth fMRI image in each time window; is the total number of local regions of the vth fMRI image in each time window; is the overall gray value of the uth local region of the vth fMRI image in each time window; is the mean value of the gray value of the pixel points in the vth fMRI image in each time window; is the variance of the gray value of the pixel points in the uth local region of the vth fMRI image in each time window; is the mean value of the elements in the slope sequence of the uth local region of the vth fMRI image in each time window; an element mean value in a slope sequence of a local region of the vth fMRI image in each time window; an element variance in a slope sequence of the uth local region of the vth fMRI image in each time window.

[0032] Further, the method for obtaining the convolution kernel size comprises:

[0033] The sum of the constant 1 and the functional connection coefficient is used to perform weighted processing on the preset convolution kernel size, and the weighted result is used as the convolution kernel size for deep learning of all fMRI images in the latter one of every two adjacent time windows.

[0034] Further, the method for deep learning of the fMRI image is a CNN model.

[0035] Further, the method for matching the local region of every two adjacent fMRI images is a template matching algorithm.

[0036] Further, the preset convolution kernel size is 9.

[0037] The present application has the following beneficial effects:

[0038] In the embodiment of the present application, because the brain region is in a continuous activation state, the similarity of the gray scale distribution and the geometric structure of the continuous fMRI images is high, therefore, according to the similarity of the geometric structure and the gray scale distribution of the matched local regions of the two adjacent fMRI images, and in combination with the image matching degree of the global image angle, the matching degree of the local regions of the two fMRI images is analyzed, and the region consistency value obtained can measure the possibility of the continuous activation state of the adjacent fMRI images, and the time window is divided by using the same, so that the dynamic evolution characteristics of the brain functional connection mode can be more accurately captured, and the analysis of the neural activity is more reliable; the difference between the region consistency values of the fMRI images in the two adjacent time windows, and the difference between the geometric structure features and the gray scale distribution features of the local regions can all reflect the possibility of the continuous activation state of the brain region in the adjacent time window, and the brain region cooperative activity degree of the adjacent time window is obtained by comprehensive analysis, and the cooperative activity degree directly reflects the stability of the brain region in space-time, so that the convolution kernel size can be adaptively adjusted based on the functional connection coefficient to maintain the space-time synchronization of the dynamic changes of the brain functional connection, so that the deep learning model can effectively capture the dynamic interaction relationship of the fMRI image in space-time, effectively solve the problem that the CNN model may be misjudged for the dynamic recombination of the brain network due to short-term when the CNN model is used for fMRI dynamic functional connection analysis, enhance the explanation of the deep learning model for complex nervous system diseases such as aphasia after stroke, and improve the analysis ability of the deep learning model for the fMRI image, and further improve the recognition effect of the local lesion in the fMRI image of the patient with aphasia after stroke. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings required to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0040] Figure 1 A step flow chart of a post-stroke aphasia patient fMRI focal recognition method based on deep learning provided by an embodiment of the present application;

[0041] Figure 2 A regional consistent value acquisition method flow chart provided by an embodiment of the present application;

[0042] Figure 3 A functional connection coefficient acquisition method flow chart provided by an embodiment of the present application;

[0043] Figure 4 A computer device schematic diagram of a post-stroke aphasia patient fMRI focal recognition device based on deep learning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined purpose, the following describes the post-stroke aphasia patient fMRI focal recognition method based on deep learning according to the present application, its specific implementation, structure, features and effects in detail in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0046] The specific scheme of the post-stroke aphasia patient fMRI focal recognition method based on deep learning provided by the present application will be specifically described below in combination with the drawings.

[0047] Embodiment 1:

[0048] The present application proposes a post-stroke aphasia patient fMRI focal recognition method based on deep learning. Please refer to Figure 1 , which shows a step flow chart of a post-stroke aphasia patient fMRI focal recognition method based on deep learning provided by an embodiment of the present application. The method comprises:

[0049] Step S1: Obtain the fMRI image sequence of the brain of the post-stroke aphasia patient, and the fMRI images in the sequence include different local regions.

[0050] Scan the brain of the post-stroke aphasia patient using a magnetic resonance device. During the scanning process, the patient performs specific tasks such as vocabulary naming, sentence repetition, and finger tapping, etc. The tasks and rest are alternated 10 times, each for 30 seconds, with 5 minutes of rest before and after. During the rest, the patient remains in a relaxed state, which can be analyzed for the dynamic connection strength of the default mode network and the language network. The magnetic resonance device collects fMRI images every 2 seconds. The fMRI images are arranged in chronological order to obtain an fMRI image sequence, which reflects the activity changes of the patient's brain when performing specific tasks or in different states.

[0051] It should be noted that, in order to reduce the head motion artifacts caused by the patient's movement during the scanning process, a special head coil and a soft pad are used to fix the head of the post-stroke aphasia patient. A magnetic resonance device with a magnetic field strength greater than 3.0 Tesla, such as Philips Ingenia 3.0T CX, is selected, and the temporal resolution is set to one layer every 2 seconds to capture the fine activation pattern of the language-related brain area.

[0052] Step S2: Match the local regions of each adjacent two fMRI images in the sequence, and obtain the regional consistency value of the corresponding two fMRI images according to the geometric structure similarity and gray distribution similarity of the two matched local regions of each adjacent two fMRI images, as well as the image matching degree; based on the regional consistency value, the corresponding time period of the sequence is divided into different time windows.

[0053] Brain functional connectivity is not static, and its dynamic evolution characteristics fluctuate with cognitive tasks, emotional states, or spontaneous activities. In order to capture the dynamic characteristics of brain functional connectivity, it is necessary to analyze the dynamic functional connectivity through time window division. When a brain region is in a sustained activation state, its fMRI signal will show a high temporal correlation in consecutive time frames, which is manifested as a regular change in local BOLD signal intensity, such as stable increase or decrease, and a relatively stable spatial range of the activated region. Among them, the BOLD signal intensity determines the gray distribution of the local region, and the spatial range determines the geometric structure of the local region. Therefore, if a brain region is in a sustained activation state, the similarity of the gray distribution and the geometric structure of the local region of consecutive fMRI images will be higher.

[0054] In order to capture the slight changes between local positions of the brain more finely, the local regions of the two adjacent fMRI images are matched to avoid the influence caused by large-scale image differences. The image matching degree of the two adjacent fMRI images is analyzed from the global image angle to analyze the similarity of the two fMRI images, and the matching degree of the local regions of the two fMRI images is analyzed in combination with the geometric structure similarity and the gray distribution similarity of the matched local regions of the two adjacent fMRI images; the greater the image matching degree, the geometric structure similarity and the gray distribution similarity, the higher the matching degree of the local regions of the two fMRI images, so that the region consistency value is obtained.

[0055] The region consistency value measures the possibility that the adjacent fMRI images are in a continuous activation state through the matching degree of the local regions of the two adjacent fMRI images, and the corresponding time period of the fMRI image sequence is divided into different time windows based on the region consistency value, so that the dynamic evolution characteristics of the brain functional connection mode can be captured more accurately, and the analysis of neural activity is more reliable. The corresponding time period of the fMRI image sequence refers to the entire time period of the patient in the task state and the resting state.

[0056] In an implementation manner of the embodiment of the application, the local regions of the two adjacent fMRI images are matched by using a template matching algorithm, and other embodiments can also use a scale-invariant feature transformation algorithm, a histogram matching algorithm and a twin neural network.

[0057] Step S3: obtaining the functional connection coefficient of the two corresponding time windows according to the difference of the region consistency value of the fMRI images in each adjacent two time windows, and the difference of the geometric structure features and the gray distribution features of the local regions.

[0058] The fMRI image reveals the dynamic changes of the brain region activity, and the brain region activity of the adjacent time windows usually has continuity and correlation. By analyzing the brain region cooperative activity degree of the adjacent time windows, the spatiotemporal coupling strength of the brain region activity is quantified, and then the evolution process is tracked. Since the cooperative activity degree directly reflects the stability of the brain region in space and time, in order to maintain the spatiotemporal synchronization of the dynamic changes of the brain functional connection, the convolution kernel size needs to be adaptively adjusted according to the brain region cooperative activity degree of the adjacent time windows, so as to optimize the sensitivity of the feature extraction of the deep learning model.

[0059] The regional consistency value measures the possibility of the adjacent fMRI images being in a sustained activation state, and the difference in the regional consistency values of adjacent time windows presents the stability of the neural activity of the brain region in adjacent time windows, and the smaller the difference is, the more stable the activity mode of the brain region in adjacent time windows is. The difference in the geometric structure features and the gray distribution features of adjacent time windows can measure the possibility of the brain region being in a sustained activation state, and the smaller the above difference is, the greater the possibility of being in a sustained activation state is, which indicates that the activity mode of the brain region in adjacent time windows is more coordinated. Therefore, the activity mode coordination degree of the brain region in adjacent time windows is analyzed by comprehensively considering the difference in the regional consistency values of the fMRI images in adjacent time windows and the difference in the spatial distribution features and the gray distribution features of the local region, and the functional connection coefficient is obtained.

[0060] Step S4: determining the convolution kernel size of the deep learning of the fMRI image in each time window according to the functional connection coefficient, and performing local lesion recognition on the fMRI image in each time window.

[0061] The functional connection coefficient measures the activity mode coordination degree of the brain region in adjacent time windows. If the activity mode coordination degree of the brain region in adjacent time windows is higher, it indicates that the brain region is more likely to form a stable coordination network, and a small-size convolution kernel needs to be used to identify the local lesion of the aphasia patient in the local details of the fMRI image, more sensitively capture the lesion features of the specific brain region of the patient, avoid the large-range blur effect, and thus more completely segment the local lesion; if the activity mode coordination degree of the brain region in adjacent time windows is lower, a large-size convolution kernel can be used to analyze the coordination abnormality of the brain function connection and capture the coordination abnormal signal across the brain region. The selection of the convolution kernel size of the time window is guided by the functional connection coefficient, which can balance the accurate positioning and wide-area coverage, the deep learning model can effectively capture the dynamic interaction relationship of the fMRI image in time and space, adapt to the diagnosis needs of different brain network states, enhance the explanation power of the deep learning model for complex nervous system diseases such as aphasia after stroke, and thus improve the analysis ability of the deep learning model for the fMRI image.

[0062] In one implementation manner of the embodiment of the present application, the deep learning method is a CNN model.

[0063] The process of local lesion recognition of the fMRI image by the CNN model through the size of the convolution kernel of deep learning according to the fMRI image in the time window is as follows: firstly, the fMRI image in the fMRI image sequence is preprocessed, such as spatial standardization, MNI space registration, time layer correction and the like, to eliminate individual differences and noise interference; secondly, a doctor labels a mask based on the anatomical position of the clinical diagnosis result such as a tumor or a seizure focus, generates a binary label graph, wherein the label value of the lesion area is 1 and the label value of the normal brain area is 0, and realizes the labeling of the local lesion area; then, a multi-level 3D convolution module is constructed according to the size of the convolution kernel corresponding to each time window, and a small size convolution kernel is preferentially used to reduce the parameter quantity and capture local subtle features such as abnormal gray matter signals; finally, the joint optimization of Dice Loss + Focal Loss is used to relieve the class imbalance problem caused by a very low proportion of local lesion areas such as a small seizure focus, and the full connection layer is fine-tuned on the basis of a pre-trained 3D medical image model such as 3D ResNet-50 on BraTS data set, so as to accelerate the convergence and improve the generalization ability in a small data scenario. It should be noted that the deep learning recognition can accurately recognize the small changes, detailed features, shapes and positions of the local lesion, which is helpful for the doctor to more accurately judge the nature of the local lesion.

[0064] Preferably, in some possible implementation manners of the embodiment of the present application, the method for obtaining the region consistent value can refer to Figure 2 The method comprises the following steps:

[0065] Step S210: a matching region pair is formed by two matching local regions of each adjacent two fMRI images; and a region overall difference value of the corresponding two fMRI images is obtained according to the difference in the gray value of the pixel points of the two local regions in all matching region pairs of each adjacent two fMRI images and the slope difference between adjacent pixel points on the edge.

[0066] In the embodiment of the present application, the method for obtaining the region overall difference value comprises: arranging the slopes between all adjacent two pixel points on the edge of the local region in sequence to obtain a slope sequence; calculating the mean value of the gray values of the pixel points in the local region as an overall gray value; calculating the product of the absolute value of the difference between the overall gray values of the two local regions in each matching region pair of each adjacent two fMRI images and the DTW value of the slope sequence, and taking the mean value of all products as the region overall difference value of the corresponding two fMRI images. The calculation formula of the region overall difference value of each adjacent two fMRI images is as follows:

[0067] ; is the region overall difference value of the cth and (c+1)th fMRI images in the fMRI image sequence. is the number of matching region pairs of the cth and (c+1)th fMRI images in the fMRI image sequence; is the overall gray value of the local region located in the cth fMRI image in the mth matching region pair of the cth and (c+1)th fMRI images in the fMRI image sequence; is the overall gray value of the local region located in the (c+1)th fMRI image in the mth matching region pair of the cth and (c+1)th fMRI images in the fMRI image sequence; is the DTW value of the slope sequence of the two local regions in the mth matching region pair of the cth and (c+1)th fMRI images in the fMRI image sequence; is an absolute value function.

[0068] It should be noted that the DTW value measures the spatial distribution similarity by analyzing the matching degree of the bending direction and the turning point distribution of the edges of the two local regions in the region matching pair, and the smaller the DTW value is, the more similar the spatial distribution of the two local regions is; the overall gray value reflects the overall gray level of the local region, and the smaller the overall gray value of the two local regions in the region matching pair is, the more similar the gray distribution of the two local regions is; the smaller the region overall difference value is, the greater the spatial distribution similarity and the gray similarity of the local regions of the adjacent two fMRI images are.

[0069] It should be noted that, in order to avoid the deviation of the DTW value caused by the arrangement of the pixel points on the edge of the local region from the random starting point, when obtaining the slope sequence, the pixel points on the edge of the local region need to be arranged in a clockwise direction from the preset point to obtain a pixel sequence, and the slopes between every two adjacent pixel points in the pixel sequence are arranged in sequence. A vertical straight line passing through the center point of the local region is drawn, and the intersection point corresponding to the maximum pixel coordinate among the intersection points of the straight line and the edge of the local region is marked as the preset point.

[0070] Step S220: obtaining a region consistency value of the two fMRI images according to the number of region matching pairs and the region overall difference value of every two adjacent fMRI images.

[0071] The number of region matching pairs of the adjacent two fMRI images reflects the image matching degree of the two fMRI images, and the more the number of region matching pairs is, the higher the matching degree of the two fMRI images is. It is known that when the brain region is in a continuous activation state, the gray distribution similarity and the geometric structure similarity of the local regions of the continuous fMRI images will be higher; when the region overall difference value is smaller and the number of region matching pairs is greater, the similarity of the geometric structure and the gray distribution of the adjacent two fMRI images is greater and the image matching degree is higher, and the adjacent two fMRI images are more similar. Therefore, the region overall difference value and the region consistency value are in a negative correlation relationship, and the number of region matching pairs and the region consistency value are in a positive correlation relationship.

[0072] In a specific implementation form of the embodiment of the present application, taking the first and second fMRI images in the fMRI image sequence as an example for analysis, the region consistency value of the first and second fMRI images is which is expressed by a formula as follows:

[0073] ;

[0074] In the formula, N is the minimum value of the number of local regions in the first and second fMRI images in the fMRI image sequence; n is the number of matching region pairs of the first and second fMRI images in the fMRI image sequence; is the region overall difference value of the first and second fMRI images in the fMRI image sequence; and exp is an exponential function with a natural constant as a base. It should be noted that, if is greater, the first and second fMRI images are more similar, and the possibility that the brain region is in a sustained activation state at the time points corresponding to the two fMRI images is greater.

[0075] Preferably, in some possible implementation forms of the embodiment of the present application, the time window division method comprises: clustering the fMRI images in the fMRI image sequence based on the region consistency value to obtain different clustering clusters; each fMRI image in the fMRI image sequence corresponds to a time point; determining whether the fMRI images in each clustering cluster are continuous in the fMRI image sequence, if yes, then taking the time points corresponding to the fMRI images with the minimum and maximum subscripts in each clustering cluster as the start time point and the end time point of a time window in sequence; if not, then dividing the fMRI images in each clustering cluster into different continuous image segments based on the continuity of the fMRI images in each clustering cluster in the fMRI image sequence, and taking the time points corresponding to the fMRI images with the minimum and maximum subscripts in each continuous image segment as the start time point and the end time point of a time window in sequence. It should be noted that the brain region is in a similar neural activity mode in the same time window.

[0076] As an example, it is assumed that the fMRI image sequence A= wherein the elements in the sequence A represent fMRI images, and the fMRI images in the clustering cluster J include ; the fMRI images in the clustering cluster J are continuous in the sequence A, , respectively, the fMRI images in the clustering cluster J are divided into , two continuous image segments.

[0077] In an implementation form of the embodiment of the present application, the DBSCAN algorithm is selected to cluster the fMRI images in the fMRI image sequence.

[0078] Preferably, in some possible implementation manners of the embodiment of the present application, the function connection coefficient acquisition method is as follows Figure 3 , which shows a function connection coefficient acquisition method flowchart provided by an embodiment of the present application, the method comprises:

[0079] Step S310: acquiring the image feature value of the corresponding time window according to the geometric structure feature and the gray distribution feature of the local region of the fMRI image in each time window.

[0080] In the embodiment of the present application, the image feature value acquisition method comprises: for each fMRI image, calculating the ratio of the absolute value of the difference between the overall gray value of each local region of the fMRI image and the mean value of the gray values of the pixel points in the fMRI image and the variance of the gray values of the pixel points in each local region, and then taking the cumulative sum of all the ratios as the gray feature value; calculating the ratio of the absolute value of the difference between the mean value of the elements in the slope sequence of each local region of the fMRI image and the mean value of the elements in the slope sequence of all local regions and the variance of the elements in the slope sequence of each local region, and then taking the cumulative sum of all the ratios as the structure feature value; taking the mean value of the product of the gray feature value and the structure feature value of all fMRI images in each time window as the image feature value of each time window. The calculation formula of the image feature value is as follows:

[0081] ; in the formula, K is the image feature value of each time window; V is the total number of fMRI images in each time window; is the gray feature value of the vth fMRI image in each time window; is the structure feature value of the vth fMRI image in each time window; is the total number of local regions of the vth fMRI image in each time window; is the overall gray value of the uth local region of the vth fMRI image in each time window; is the mean value of the gray values of the pixel points in the vth fMRI image in each time window; is the variance of the gray values of the pixel points in the uth local region of the vth fMRI image in each time window; is the mean value of the elements in the slope sequence of the uth local region of the vth fMRI image in each time window; is the mean value of the elements in the slope sequence of the local region of the vth fMRI image in each time window; is the variance of the elements in the slope sequence of the uth local region of the vth fMRI image in each time window.

[0082] ​​It should be noted that the gray scale feature value and the structure feature value simultaneously quantify the local and global contrast intensity and regional consistency of the fMRI image, significantly improve the sensitivity of the gray scale distribution feature and the geometric structure feature to abnormal regions and unique structures, and are suitable for lesion segmentation or dynamic functional connection research which needs high-precision positioning of medical image analysis. The high gray scale feature value indicates that the global gray scale contrast of the fMRI image is high and the regional uniformity is high, and the high structure feature value indicates that the local regional edge pattern of the fMRI image is significantly different from the global structure and is stable itself; the overall level of the gray scale distribution feature and the geometric structure feature of all fMRI images in the analysis time window is obtained, and the greater the image feature value indicates that the global gray scale contrast of the fMRI image in the time window is high and the regional uniformity is high, and the global structure difference is significant and stable itself.

[0083] In other embodiments of the application, the gray scale value variance of the pixel points in each local region of the fMRI image can be calculated, and the mean value of all variances is taken as the gray scale feature value; the element variance in the slope sequence of each local region of the fMRI image is calculated, and the mean value of all variances is taken as the structure feature value.

[0084] Step S320: Calculate the mean value of the regional consistency values of all fMRI images in each time window, and perform negative correlation and normalization processing on the absolute value of the difference between the mean values of each adjacent two time windows and the absolute value of the difference between the image feature values, to obtain the functional connection coefficient corresponding to the two time windows.

[0085] The calculation formula of the functional connection coefficient of each adjacent two time windows is as follows:

[0086] In the formula, is the functional connection coefficient of the dth and d+1th time window; is the mean value of the regional consistency values of all adjacent two fMRI images in the dth time window; is the mean value of the regional consistency values of all adjacent two fMRI images in the d+1th time window; is the image feature value of the dth time window; is the image feature value of the d+1th time window; exp is the exponential function with the natural constant as the base; is the absolute value function.

[0087] The mean value of the regional consistency values of all fMRI images in the time window presents the overall stability of the neural activity of the time window, if the difference between the mean values of two adjacent time windows is smaller, the neural activity of the two time windows is more stable, and the coordination degree of the brain region activity patterns in the two time windows is higher; the image feature value presents the overall level of the gray scale distribution feature and the geometric structure feature of the fMRI image in the time window, if the difference between the image feature values of two adjacent time windows is smaller, the difference between the geometric structure features and the gray scale distribution features of the adjacent time windows is smaller, and the possibility that the brain region is in a continuous activation state in the two time windows is larger, the coordination degree of the brain region activity patterns in the two time windows is higher. Therefore, the absolute value of the difference between the mean value of the regional consistency values of each adjacent two time windows and the absolute value of the difference between the image feature values are multiplied to obtain the functional connection coefficient. If the functional connection coefficient is larger, the coordination degree of the brain region activity patterns in the two adjacent time windows is higher, the functional connection of the brain region in the two time windows is more stable, and the continuity of the functional network state is better; if the functional connection coefficient is smaller, it indicates that the functional connectivity of the brain region in the two adjacent time windows is worse, which may correspond to the mutation of the brain network state such as task switching, pathological activity and the like.

[0088] It should be noted that, in the embodiment of the present application, the normalization processing is performed by using Norm, and other normalization methods such as function transformation, maximum and minimum normalization and the like can also be selected, which are not limited herein.

[0089] Preferably, in some possible implementation manners of the embodiment of the present application, the method for obtaining the convolution kernel size comprises: performing weighted processing on the preset convolution kernel size by using the sum of the constant 1 and the functional connection coefficient, and taking the weighted result as the convolution kernel size for deep learning of all fMRI images in the latter one of each adjacent two time windows. It should be noted that, if the functional connection coefficient of the two adjacent time windows is larger, it indicates that the coordination degree of the brain region activity patterns in the two time windows is higher, and the brain region is more likely to form a stable collaborative network, and a small size convolution kernel is required to identify the local lesion of the aphasia patient in the local details of the fMRI image, and then the functional connection coefficient after the negative correlation and normalization processing is used to adjust the preset convolution kernel size to determine the convolution kernel size of the time window.

[0090] In order to avoid the sliding offset problem of the convolution kernel caused by the fact that the center anchor point cannot be determined due to the even size, it is necessary to ensure that the convolution kernel size is odd, and the specific method is as follows: when the integer part of the weighted result is odd, the result of taking the weighted result down to the nearest integer is taken as the convolution kernel size for deep learning of the fMRI images in the latter one of each adjacent two time windows; when the integer part of the weighted result is even, the result of taking the weighted result up to the nearest integer is taken as the convolution kernel size for deep learning of the fMRI images in the latter one of each adjacent two time windows.

[0091] In one implementation form of the embodiment of the application, the preset convolution kernel size is set to 9.

[0092] It should be noted that the convolution kernel size is a general term of the length and width of the convolution kernel, and the length and width of the convolution kernel are equal in the embodiment, and the value range of the convolution kernel size is an odd number between 1 and 9. The convolution kernel sizes of the fMRI images in the first two time windows for deep learning are equal.

[0093] Thus far, the application is completed.

[0094] Embodiment 2

[0095] The application further provides a computer device schematic diagram of the fMRI local lesion recognition device for aphasia patients after stroke based on deep learning, please refer to Figure 4 The computer device includes a memory 501, a processor 502, and a computer program 503 stored in the memory 501 and running on the processor 502, wherein when the processor 502 executes the computer program 503, the computer device can execute any one of the foregoing introduction of the fMRI local lesion recognition method for aphasia patients after stroke based on deep learning.

[0096] In addition, the embodiment of the application also protects a device, which can include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute the fMRI local lesion recognition method for aphasia patients after stroke based on deep learning provided by the embodiment of the application.

[0097] The embodiment can divide the device into functional modules according to the above-mentioned method examples, for example, each functional module can be corresponding, or two or more functions can be integrated in one processing module, and the integrated module can be realized in the form of hardware. It should be noted that the division of the modules in the embodiment is illustrative, and is only a logical function division, and another division mode can be used in actual implementation.

[0098] In the case of dividing each module corresponding to each function, the device can further include a communication module, a signal analysis module, a complexity analysis module, and a positioning module, etc. It should be noted that all related contents of each step involved in the above method embodiment can be referred to the function description of the corresponding functional module, and will not be repeated here.

[0099] It should be understood that the device provided by the embodiment is used to execute the above-mentioned fMRI local lesion recognition method for aphasia patients after stroke based on deep learning, and thus the same effect as the above-mentioned implementation method can be achieved.

[0100] In the case of employing the integrated unit, the device can include a processing module, a storage module. Wherein, when the device is applied to the equipment, the processing module can be used to control and manage the actions of the equipment. The storage module can be used to support the equipment to execute mutual program codes and the like.

[0101] Wherein, the processing module can be a processor or a controller, which can realize or execute various exemplary logical blocks, modules and circuits contained in combination with the disclosure of the present application. The processor can also be a combination of realizing computing functions, such as including one or more microprocessor combinations, a combination of digital signal processing (Digital Signal Processing, DSP) and microprocessor, and the like. The storage module can be a memory.

[0102] Embodiment 3:

[0103] The embodiment also provides a computer readable storage medium, which stores computer program codes, when the computer program codes are run on the computer, the computer executes the above-mentioned related method steps to realize the stroke aphasia patient fMRI focal recognition method based on deep learning provided by the above-mentioned embodiment.

[0104] Embodiment 4:

[0105] The embodiment also provides a computer program product, when the computer program product is run on the computer, the computer executes the above-mentioned related steps to realize the stroke aphasia patient fMRI focal recognition method based on deep learning provided by the above-mentioned embodiment.

[0106] Wherein, the device, the computer readable storage medium, the computer program product or the chip provided by the embodiment are used to execute the corresponding method provided above, so the beneficial effects that can be achieved are referred to the beneficial effects in the corresponding method provided above, which will not be repeated here.

[0107] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual ones can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0108] It is to be noted that the sequential order of the above-described embodiments of the present application only for the purpose of description, but not the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0109] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments.

Claims

1. A method for identifying a lesion in a post-stroke aphasia patient based on deep learning of fMRI, characterized by, The method comprises: obtaining an fMRI image sequence of a brain of a post-stroke aphasia patient, the fMRI images in the sequence comprising different local regions; matching the local regions of each two adjacent fMRI images in the sequence, obtaining a region consistency value of the two corresponding fMRI images according to geometric structure similarity and grayscale distribution similarity of the two matched local regions of each two adjacent fMRI images, and image matching degree, and dividing a corresponding time period of the sequence into different time windows based on the region consistency value; obtaining a functional connection coefficient of the two corresponding time windows according to a difference of the region consistency values of the fMRI images in each two adjacent time windows, and a difference of geometric structure features and grayscale distribution features of the local regions; determining a convolution kernel size for deep learning of the fMRI images in each time window according to the functional connection coefficient, specifically, adjusting a preset convolution kernel size to determine the convolution kernel size of the time window by using the functional connection coefficient after negative correlation and normalization processing, and performing focal recognition on the fMRI images in each time window; the obtaining of the functional connection coefficient of the two corresponding time windows comprises: obtaining an image feature value of the corresponding time window according to the geometric structure features and the grayscale distribution features of the local regions of the fMRI images in each time window; calculating a mean value of the region consistency values of all fMRI images in each time window, and performing negative correlation and normalization processing on a product of an absolute value difference of the mean value and an absolute value difference of the image feature value of each two adjacent time windows, to obtain the functional connection coefficient of the two corresponding time windows; a calculation formula of the functional connection coefficient of each two adjacent time windows is as follows: ; wherein, is the functional connection coefficient of the d, d+1th time window; is the mean value of the region consistency values of all adjacent two fMRI images in the dth time window; is the mean value of the region consistency values of all adjacent two fMRI images in the d+1th time window; is the image feature value of the dth time window; is the image feature value of the d+1th time window; exp is the exponential function with the natural constant as the base number; is the absolute value function. 2.The deep learning-based post-stroke aphasia patient fMRI localization method according to claim 1, wherein, the obtaining of the region consistency value of the two corresponding fMRI images comprises: forming a matching region pair from the two matched local regions of each two adjacent fMRI images, and obtaining a region overall difference value of the two corresponding fMRI images according to a grayscale value difference of pixel points of the two local regions in all matching region pairs of each two adjacent fMRI images and a slope difference between adjacent pixel points on an edge; obtaining the region consistency value of the two fMRI images according to the number of region matching pairs and the region overall difference value of each two adjacent fMRI images. 3.The deep learning-based post-stroke aphasia patient fMRI localization method according to claim 2, characterized in that, the obtaining of the region overall difference value of the two corresponding fMRI images comprises: sequentially arranging slopes between all adjacent two pixel points on an edge of a local region to obtain a slope sequence, and calculating a mean value of grayscale values of pixel points in the local region as an overall grayscale value; calculating a product of an absolute value difference of the overall grayscale value of the two local regions in each matching region pair of each two adjacent fMRI images and a DTW value of the slope sequence, and taking a mean value of all products as the region overall difference value of the two corresponding fMRI images; a calculation formula of the region overall difference value of each two adjacent fMRI images is as follows: ; is the region overall difference value of the c, c+1th fMRI image in the fMRI image sequence; is the number of matching region pairs of the c, c+1th fMRI image in the fMRI image sequence; is the overall gray value of the local region located in the cth fMRI image in the mth matching region pair of the c, c+1th fMRI image in the fMRI image sequence; is the overall gray value of the local region located in the c+1th fMRI image in the mth matching region pair of the c, c+1th fMRI image in the fMRI image sequence; is the DTW value of the slope sequence of the two local regions in the mth matching region pair of the c, c+1th fMRI image in the fMRI image sequence; is the absolute value function. 4.The deep learning-based post-stroke aphasia patient fMRI localization method according to claim 1, wherein, the dividing of the corresponding time period of the sequence into different time windows based on the region consistency value comprises: clustering the fMRI images in the fMRI image sequence based on the region consistency value to obtain different clustering clusters; Each fMRI image in the fMRI image sequence corresponds to a time point; whether the fMRI images in each cluster are continuous in the fMRI image sequence is determined, if yes, the time points corresponding to the fMRI images with the smallest and largest subscripts in each cluster are taken as the start time and end time of a time window, respectively; If not, the fMRI images in each cluster are divided into different continuous image segments based on the continuity of the fMRI images in each cluster in the fMRI image sequence, and the time points corresponding to the fMRI images with the smallest and largest subscripts in each continuous image segment are taken as the start time and end time of a time window, respectively. 5.The deep learning-based post-stroke aphasia patient fMRI localization method according to claim 3, wherein, The method for obtaining the image feature value of the corresponding time window comprises: For each fMRI image, the ratio of the absolute value of the difference between the overall gray value of each local region of the fMRI image and the mean value of the gray values of the pixel points in the fMRI image to the variance of the gray values of the pixel points in each local region is calculated, and the cumulative sum of all ratios is taken as the gray feature value; The ratio of the absolute value of the difference between the mean value of the elements in the slope sequence of each local region of the fMRI image and the mean value of the elements in the slope sequence of all local regions to the variance of the elements in the slope sequence of each local region is calculated, and the cumulative sum of all ratios is taken as the structure feature value; The mean value of the product of the gray feature value and the structure feature value of all fMRI images in each time window is taken as the image feature value of each time window; the calculation formula of the image feature value is as follows: , , ; wherein K is the image feature value of each time window; V is the total number of fMRI images in each time window; is the gray scale feature value of the vth fMRI image in each time window; is the structure feature value of the vth fMRI image in each time window; is the total number of local regions of the vth fMRI image in each time window; is the overall gray scale value of the u local region of the vth fMRI image in each time window; is the mean of the gray scale values of the pixels in the vth fMRI image in each time window; is the variance of the gray scale values of the pixels in the u local region of the vth fMRI image in each time window; is the mean of the elements in the slope sequence of the u local region of the vth fMRI image in each time window; is the mean of the elements in the slope sequence of the local region of the vth fMRI image in each time window; is the variance of the elements in the slope sequence of the u local region of the vth fMRI image in each time window. 6.The deep learning-based post-stroke aphasia patient fMRI localization method of claim 1, wherein, The method for obtaining the convolution kernel size comprises: The sum of the constant 1 and the functional connection coefficient is used to weight the preset convolution kernel size, and the weighted result is taken as the convolution kernel size for deep learning of all fMRI images in the latter one of each adjacent two time windows. 7.The deep learning-based post-stroke aphasia patient fMRI localization method of claim 1, wherein, The CNN model is used for deep learning of the fMRI images. 8.The deep learning-based post-stroke aphasia patient fMRI localization method of claim 1, wherein, The method for matching the local regions of each adjacent two fMRI images is a template matching algorithm. 9.The deep learning-based post-stroke aphasia patient fMRI localization method of claim 6, wherein, The preset convolution kernel size is 9.

Images