Spatial and temporal feature extraction method and system based on MRI image

Abstract

The invention provides a spatial-temporal feature extraction method and system based on an MRI image, and the method comprises the steps: carrying out the preliminary feature extraction of an MRI image sequence, and obtaining the initial features of the MRI image; weighting the initial features based on a time sequence attention mechanism, highlighting the features of key time steps, further optimizing feature representation, and obtaining optimized features; according to the dynamic time convolution, a convolution kernel is dynamically adjusted, optimization features are further extracted and refined at different time and spatial scales, and deep space-time features of the MRI image are obtained; according to the method, the initial features are extracted from the initial features, then the initial features are subjected to optimized feature representation by adopting a time sequence attention mechanism, and finally, deep feature extraction and refinement are performed through dynamic time convolution, so that the space-time information in the MRI image can be fully mined, the description capability of a complex structure and dynamic change in the MRI image is improved, and the accuracy of the MRI image is improved. And the MRI image spatial-temporal feature extraction performance is effectively improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and specifically to a spatio-temporal feature extraction method and system based on MRI images. Background Art

[0002] MRI images (Magnetic Resonance Imaging) are medical images generated using the principle of nuclear magnetic resonance, which can clearly display the internal structure of the human body, including organs, tissues, blood vessels, etc. MRI images use specific magnetic fields and radiofrequency pulses to cause hydrogen atomic nuclei in the body to resonate, release signals, and form images after being processed by a computer.

[0003] Currently, there are the following problems in feature extraction of MRI images: Traditional MRI image feature extraction methods, such as manual feature extraction methods (such as HOG, SIFT, etc.), often can only extract local and shallow features of the image, and cannot fully mine the rich semantic information and structural information in MRI images; moreover, the design of these manual features depends on prior knowledge and experience, and it is difficult to design manual features that can comprehensively and accurately describe image features for complex MRI image data.

[0004] Some traditional MRI image analysis methods do not fully consider the temporal characteristics of MRI image sequences, treat the image sequences as independent image sets for processing, and ignore the temporal dependence relationships between images, resulting in the inability to capture the changing rules and dynamic information of physiological processes or disease developments over time. Even if some methods consider time series information, they often adopt simple time averaging or sequential processing methods and cannot effectively highlight the importance of key time steps.

[0005] Many traditional image feature extraction methods, including some methods based on convolutional neural networks, usually use convolutional kernels of fixed size and shape to extract features, and can only capture image features at a fixed spatial scale. For target structures or lesion regions of different sizes and shapes, it is difficult to adaptively adjust the convolutional kernels to obtain the best feature representation. When processing MRI images, due to different tissue structures and lesions may have different spatial scales and temporal change speeds, fixed-scale feature extraction methods may not be able to effectively capture these multi-scale information.

[0006] In summary, currently, feature extraction of MRI images faces many challenges, and new methods and technologies need to be continuously explored to overcome these problems. Summary of the Invention

[0007] In order to overcome the above problems existing in the prior art, this application provides a spatio-temporal feature extraction method and system based on MRI images, and adopts the following technical solutions:

[0008] In a first aspect, the present application provides a spatio-temporal feature extraction method based on MRI images, including:

[0009] Perform preliminary feature extraction on the MRI image sequence to obtain the initial features of the MRI image;

[0010] Based on the temporal attention mechanism, weight the initial features to highlight the features of key time steps, further optimize the feature representation, and obtain optimized features;

[0011] Dynamic time convolution further extracts and refines the optimized features at different time and space scales by dynamically adjusting the convolution kernel, and obtains the deep spatio-temporal features of the MRI image.

[0012] Furthermore, perform preliminary feature extraction on the MRI image sequence through an LSTM-GRU hybrid network to obtain the initial features of the MRI image.

[0013] Furthermore, before the MRI image enters the LSTM-GRU hybrid network, preprocess the MRI image, and the preprocessing includes:

[0014] Normalize the gray value of the MRI image to a preset range, and adjust the size of the MRI image so that the MRI image has the same spatial resolution and size.

[0015] Furthermore, after preprocessing the MRI image before it enters the LSTM-GRU hybrid network, it also includes:

[0016] Input the preprocessed MRI image into the first convolutional layer of the convolutional neural network;

[0017] The convolution kernel slides on the image according to the set stride, and performs convolution operations on each local area; each convolution kernel generates a corresponding feature map, and multiple convolution kernels operate in parallel to obtain multiple feature maps;

[0018] Repeat the above convolution process, use the output of the previous convolutional layer as the input of the next convolutional layer, and gradually extract different levels of features of the image through the stacking of multiple convolutional layers;

[0019] After passing through several convolutional layers and activation function layers, insert a pooling layer, divide the feature map into several non-overlapping local areas, and take the maximum value in each local area as the result after pooling to obtain the downsampled feature map;

[0020] Fuse the feature maps after multiple convolution, activation, and pooling operations, and use the fused features as the output of the convolutional neural network.

[0021] Further, the fused features are used as the input of the LSTM-GRU hybrid network to perform preliminary feature extraction on the MRI image sequence, and the initial features of the MRI image are obtained, including:

[0022] Arrange the fused features of multiple MRI images in chronological order to form a sequence;

[0023] Input the feature sequence with adjusted format into the LSTM layer. The LSTM layer updates the cell state and the hidden state according to the current input and the hidden state of the previous moment; after being processed by the LSTM, the long-term dependence information in the sequence is encoded into the hidden state;

[0024] Use the output of the LSTM layer as the input of the GRU layer; the GRU layer quickly captures the short-term changes in the sequence, further processes the hidden state output by the LSTM, and extracts the short-term feature information of the sequence;

[0025] After being processed by the LSTM and the GRU, take the hidden state of the last time step as the initial feature representation of the MRI image sequence.

[0026] Further, based on the temporal attention mechanism, weight the initial features to highlight the features of the key time steps, further optimize the feature representation, and obtain the optimized features, including:

[0027] For the initial feature sequence X = [x 1 , x 2 ,..., x T input by the temporal attention mechanism, where T is the number of time steps, and x t is the feature vector of the t-th time step, and its dimension is d;

[0028] Map the input initial feature sequence to three spaces of query Q, key K, and value V;

[0029] Calculate the similarity between the query and the key to obtain the attention score;

[0030] Convert the attention score into a probability distribution to obtain the attention weight;

[0031] Based on the calculated attention weight, perform a weighted sum on the value vectors to obtain the optimized feature of each time step where v i is the value vector of the i-th time step; finally, obtain the optimized feature sequence

[0032] Further, convert the attention score into a probability distribution to obtain the attention weight, including:

[0033] Convert the attention scores into a probability distribution and normalize using the softmax function: where α t,i is the attention weight of the t-th time step to the i-th time step, satisfying

[0034] In a second aspect, the present application also provides a spatio-temporal feature extraction system based on MRI images, including:

[0035] An initial feature acquisition module for performing preliminary feature extraction on the MRI image sequence to obtain the initial features of the MRI image;

[0036] An optimized feature acquisition module for weighting the initial features based on the temporal attention mechanism to highlight the features of the key time steps, further optimizing the feature representation, and obtaining optimized features;

[0037] A deep spatio-temporal feature acquisition module for dynamically adjusting the convolutional kernel through dynamic time convolution to further extract and refine the optimized features at different time and space scales, and obtaining the deep spatio-temporal features of the MRI image.

[0038] In a third aspect, the present application provides an electronic device, including:

[0039] One or more processors; a memory; and one or more computer programs, where the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to execute the method described in the first aspect.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the method described in the first aspect.

[0041] In a fifth aspect, the present application provides a computer program that, when executed by a computer, is used to execute the method described in the first aspect.

[0042] In a possible design, the program in the fifth aspect can be stored in whole or in part on a storage medium packaged together with the processor, or can be stored in whole or in part on a memory not packaged together with the processor.

[0043] The present application has the following beneficial effects:

[0044] 1. This application performs preliminary feature extraction on the MRI image sequence through an LSTM-GRU hybrid network to obtain the initial features of the MRI image. By performing preliminary feature extraction on the MRI image sequence, this application can utilize the continuity and correlation of the MRI image sequence in the time dimension to extract richer and more comprehensive initial features than single-image features, avoiding the problem of missing feature information caused by only relying on single images or simple handcrafted feature extraction methods.

[0045] 2. This application weights the initial features based on a temporal attention mechanism to highlight the features of key time steps, further optimizing the feature representation to obtain optimized features; by introducing a temporal attention mechanism to weight the initial features, the temporal attention mechanism can automatically learn the importance weights of the features at each time step, highlighting the features of key time steps and suppressing unimportant information, thereby further optimizing the feature representation; this application pays more attention to the parts with key information in the image sequence, better utilizes the time series information to optimize the feature representation, and thus more accurately describes the changes of MRI images over time, contributing to the discovery of some lesions or physiological phenomena with specific change patterns over time.

[0046] 3. This application further extracts and refines the optimized features at different time and space scales through dynamic time convolution by dynamically adjusting the convolutional kernel to obtain the deep spatio-temporal features of the MRI image; by dynamically adjusting the convolutional kernel, this application adaptively captures richer and more refined features at different time and space scales according to the characteristics and data distribution of the MRI image, thereby obtaining the deep spatio-temporal features of the MRI image, which helps to more accurately analyze and understand the complex information in the MRI image, thus improving the description ability of complex structures and dynamic changes in the MRI image.

[0047] 4. This application forms a complete and systematic spatio-temporal feature extraction framework for MRI images by improving and deepening each step from obtaining initial features through preliminary feature extraction, to optimizing feature representation using a temporal attention mechanism, and then to performing deep feature extraction and refinement through dynamic time convolution. This framework can fully exploit the spatio-temporal information in MRI images and effectively improve the performance of spatio-temporal feature extraction for MRI images. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is an exemplary system architecture diagram to which the embodiments of this application can be applied;

[0049] Figure 2 It is a flowchart of the spatio-temporal feature extraction method based on MRI images according to the embodiments of this application;

[0050] Figure 3Flow chart for obtaining input features of the LSTM-GRU hybrid network according to an embodiment of the present application;

[0051] Figure 4 Flow chart for extracting initial features of MRI images according to an embodiment of the present application;

[0052] Figure 5 System flow chart according to an embodiment of the present application;

[0053] Figure 6 Schematic diagram of a computer device according to an embodiment of the present application. Detailed implementation manners

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs; the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this application or the above drawings are used to distinguish different objects and not to describe a specific order.

[0055] Reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0056] In order to enable those skilled in the technical field to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0057] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0058] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0059] Terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, and so on.

[0060] Server 105 can be a server that provides various services, such as a background server that provides support for the pages displayed on terminal devices 101, 102, and 103.

[0061] It should be noted that the method for extracting spatio-temporal features based on MRI images provided in the embodiments of the present application is generally executed by a server / terminal device. Correspondingly, the system for extracting spatio-temporal features based on MRI images is generally set in a server / terminal device.

[0062] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0063] Continue to refer to Figure 2 , which shows a flowchart of a method for extracting spatio-temporal features based on MRI images according to the present application. The method includes the following steps:

[0064] Step 201, perform preliminary feature extraction on the MRI image sequence to obtain the initial features of the MRI image.

[0065] In a possible implementation manner, the present application performs preliminary feature extraction on the MRI image sequence through an LSTM-GRU hybrid network to obtain the initial features of the MRI image.

[0066] In a possible implementation manner, before the MRI image enters the LSTM-GRU hybrid network, preprocess the MRI image. The preprocessing includes:

[0067] Normalize the gray values of the MRI image to a preset range, and adjust the size of the MRI image so that the MRI image has the same spatial resolution and dimensions; normalize the gray values of the MRI image to a preset range such as [0, 1] or [-1, 1] to eliminate the gray value differences caused by different devices or scanning parameters. Commonly used methods include min-max normalization, Z-score normalization, etc. Adjust the size of the MRI image so that the MRI image has the same spatial resolution and dimensions. An interpolation algorithm can be used, such as bilinear interpolation, cubic spline interpolation, etc., to scale the image to the fixed size required by the network input.

[0068] In a possible implementation, before the MRI image enters the LSTM-GRU hybrid network, after preprocessing the MRI image, it further includes:

[0069] Step 31, input the preprocessed MRI image into the first convolutional layer of the convolutional neural network.

[0070] Step 32, the convolutional kernel slides on the image according to the set stride, and performs a convolutional operation on each local area, that is, performs a weighted sum of the weights of the convolutional kernel and the pixel values of the local area of the image, and adds a bias term to obtain the output feature map of the convolutional layer. Each convolutional kernel will generate a corresponding feature map. Multiple convolutional kernels operate in parallel, and multiple feature maps can be obtained. These feature maps respectively represent the responses of the image in different feature dimensions.

[0071] Step 33, repeat the above convolutional process, take the output of the previous convolutional layer as the input of the next convolutional layer, and gradually extract different levels of features of the image through the stacking of multiple convolutional layers.

[0072] In a possible implementation, after each convolutional layer, input the output result of the convolutional layer into an activation function (such as the ReLU function) for non-linear transformation. Through the non-linear mapping of the activation function in this application, the convolutional neural network can learn more complex non-linear relationships, enhance the representation ability of the network, and thus be able to extract richer and more discriminative image features.

[0073] Step 34, after passing through several convolutional layers and activation function layers, insert a pooling layer, divide the feature map into several non-overlapping local areas, and take the maximum value in each local area as the result after pooling to obtain the downsampled feature map. Through the pooling operation in this application, the spatial dimension of the feature map can be reduced, the amount of data and the amount of calculation can be reduced, while retaining the most important feature information, improving the robustness and stability of the features, and helping to prevent overfitting.

[0074] Step 35: Fuse the feature maps after multiple convolution, activation, and pooling operations, and use the fused features as the output of the convolutional neural network. By fusing the feature maps after multiple convolution, activation, and pooling operations, this application can combine features of different levels and types to form a more comprehensive and representative feature representation. This application uses the fused features as the input to the LSTM-GRU hybrid network, providing rich image feature information for time series modeling by LSTM and GRU.

[0075] In a possible implementation, use the fused features as the input to the LSTM-GRU hybrid network to perform preliminary feature extraction on the MRI image sequence and obtain the initial features of the MRI image. Please refer to Figure 4 , and the specific content includes:

[0076] Step 41: Arrange the fused features of multiple MRI images in chronological order to form a sequence. Assume that each MRI image obtains a feature vector of length n after being processed by the convolutional neural network. For an MRI image sequence with T time steps, the shape of the sequence data input to the LSTM-GRU is (T, n). To improve the training efficiency, multiple sequences are usually grouped into a batch for processing; if the batch size is B, the shape of the input data becomes (B, T, n).

[0077] Step 42: Input the feature sequence with adjusted format into the LSTM layer. The LSTM layer updates the cell state and hidden state based on the current input and the hidden state at the previous moment. After being processed by the LSTM, the long-term dependence information in the sequence is encoded into the hidden state. For example, when analyzing a cardiac MRI image sequence, the LSTM can remember the morphological changes of the heart at different time points, thereby extracting long-term features related to cardiac function.

[0078] Step 43: Use the output of the LSTM layer as the input to the GRU layer. The GRU layer quickly captures the short-term changes in the sequence and further processes the hidden state output by the LSTM to extract the short-term feature information of the sequence.

[0079] Step 44: After being processed by the LSTM and GRU, take the hidden state at the last time step as the initial feature representation of the MRI image sequence. This initial feature representation contains both long-term dependence information and short-term change information.

[0080] Among them, the LSTM-GRU hybrid network is a deep learning model that combines the advantages of the long short-term memory network (LSTM) and the gated recurrent unit (GRU) and is used to process sequence data. The LSTM is utilized to capture the long-term dependencies in the MRI image sequence, and the GRU is used to process the short-term rapid change features, enabling the network to simultaneously model and extract the spatio-temporal features of the MRI image at different time scales and more comprehensively describe the spatio-temporal information in the MRI image. The method combined with the LSTM-GRU hybrid network and others in this application is based on the automatic feature learning mechanism of deep learning, which can automatically learn more advanced and abstract spatio-temporal features from a large amount of MRI image data without manual feature design, greatly improving the efficiency and accuracy of feature extraction. The method combined with the LSTM-GRU hybrid network and others can learn the spatio-temporal feature patterns of MRI images in various situations through training on a large-scale MRI image dataset, has good adaptability and generalization ability for MRI images from different sources and with different characteristics, and can better handle various complex situations in actual clinical applications.

[0081] Step 202: Weight the initial features based on the temporal attention mechanism to highlight the features of key time steps, further optimize the feature representation, and obtain optimized features.

[0082] In the embodiment of this application, the temporal attention mechanism can assign different weights according to the importance of features at different time steps, can pay more attention to the features of key time steps, suppress irrelevant information, and thus optimize the feature representation; in the MRI image sequence, the temporal attention mechanism can highlight the key time points related to diseases and further improve the expression ability of features.

[0083] In a possible implementation manner, weighting the initial features based on the temporal attention mechanism to highlight the features of key time steps, further optimizing the feature representation, and obtaining optimized features includes:

[0084] For the initial feature sequence X = [x 1 , x 2 ,..., x T input to the temporal attention mechanism, where T is the number of time steps, and x t is the feature vector at the t-th time step, and its dimension is d.

[0085] Map the input initial feature sequence to three spaces of query Q, key K, and value V; in the temporal attention mechanism, usually, the query, key, and value are obtained based on the input feature sequence X. The mapping is achieved through linear transformation, where Q = XW Q , K = XW K , V = XW V , where W Q 、WK and W V are learnable weight matrices for query Q, key K, and value V, with dimensions d×d q , d×d k and d×d v Typically, d q =d k .

[0086] Calculate the similarity between the query and the key to obtain the attention score. The calculation method is: e t,i =q t T k i , where q t is the query vector at the t-th time step, and k i is the key vector at the i-th time step. e t,i represents the attention score of the t-th time step to the i-th time step.

[0087] Convert the attention score into a probability distribution and normalize it using the softmax function: where α t,i is the attention weight of the t-th time step to the i-th time step, satisfying

[0088] Based on the calculated attention weights, perform a weighted sum on the value vectors to obtain the optimized features for each time step where v i is the value vector at the i-th time step; finally, obtain the optimized feature sequence

[0089] Step 203, the dynamic temporal convolution further extracts and refines the optimized features at different temporal and spatial scales by dynamically adjusting the convolution kernel, and obtains the deep spatio-temporal features of the MRI image.

[0090] In a possible implementation, the dynamic temporal convolution further extracts and refines the optimized features at different temporal and spatial scales by dynamically adjusting the convolution kernel, and obtains the deep spatio-temporal features of the MRI image, including:

[0091] Dynamically generate a convolution kernel according to the input optimized features through a meta-network of the dynamic temporal convolution. The meta-network can be a simple multi-layer perceptron (MLP). The meta-network takes the input features as input and outputs the parameters of the convolution kernel. For example, for a one-dimensional dynamic temporal convolution, the meta-network can map the input features to a convolution kernel parameter vector, which contains the weights and biases of the convolution kernel.

[0092] Perform a convolution operation on the input optimized features using dynamically generated convolutional kernels; for the features at each time step, use the convolutional kernels generated at the corresponding time step for convolution; the convolution operation is expressed as: where y t is the output feature at the t-th time step, w t,i is the i-th weight of the convolutional kernel generated at the t-th time step, n is the size of the convolutional kernel, and x t+i is the feature at the (t + i)-th time step in the input feature sequence.

[0093] After the convolution operation, introduce non-linearity through a non-linear activation function to enhance the expressive power of the dynamic time convolution model. The non-linear activation function is expressed as: z t = RELU(y t ), where z t is the feature at the t-th time step processed by the activation function.

[0094] After dynamic time convolution and non-linear activation, the obtained feature sequence Z = [z 1 , z 2 ,..., z T is the deep spatio-temporal feature of the MRI image. These feature sequences contain information extracted and refined at different time and space scales, and can more comprehensively describe the physiological and pathological features in the MRI image sequence.

[0095] This application combines the LSTM-GRU hybrid network, the temporal attention mechanism, and dynamic time convolution to extract and optimize the spatio-temporal features of MRI images from different perspectives and levels; by adopting a multi-level feature extraction method, it can effectively capture the complex spatio-temporal information in the MRI image sequence and provide more accurate and valuable information for medical diagnosis and research.

[0096] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. This computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), etc., or a Random Access Memory (RAM), etc.

[0097] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit and can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0098] Continue to refer to Figure 5 , the spatio-temporal feature extraction system based on MRI images described in this embodiment includes:

[0099] An initial feature acquisition module 501, configured to perform preliminary feature extraction on an MRI image sequence to obtain initial features of the MRI image;

[0100] An optimized feature acquisition module 502, configured to weight the initial features based on a temporal attention mechanism, highlight the features of key time steps, further optimize the feature representation, and obtain optimized features;

[0101] A deep spatio-temporal feature acquisition module 503, configured to perform dynamic time convolution to further extract and refine the optimized features at different time and space scales by dynamically adjusting the convolution kernel, and obtain deep spatio-temporal features of the MRI image.

[0102] To solve the above technical problems, the embodiments of the present application also provide a computer device. Specifically, please refer to Figure 6 , Figure 6 which is the basic structural block diagram of the computer device in this embodiment.

[0103] The computer device 6 includes a memory 6a, a processor 6b, and a network interface 6c that are communicatively connected to each other through a system bus. It should be noted that only the computer device 6 with components 6a-6c is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented. Among them, those skilled in the art of the present technology can understand that a computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0104] The computer device may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touchpad, a voice control device, or the like.

[0105] The memory 6a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 6a may be an internal storage unit of the computer device 6, such as the hard disk or the memory of the computer device 6. In other embodiments, the memory 6a may also be an external storage device of the computer device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a FlashCard, etc. equipped on the computer device 6. Of course, the memory 6a may also include both the internal storage unit and the external storage device of the computer device 6. In this embodiment, the memory 6a is generally used to store the operating system and various application software installed on the computer device 6, such as the program code of the method for extracting spatio-temporal features based on MRI images. In addition, the memory 6a can also be used to temporarily store various types of data that have been output or will be output.

[0106] In some embodiments, the processor 6b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 6b is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 6b is used to run the program code stored in the memory 6a or process data, such as running the program code of the method for extracting spatio-temporal features based on MRI images.

[0107] The network interface 6c may include a wireless network interface or a wired network interface, and the network interface 6c is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0108] The present application also provides another implementation manner, that is, to provide a non-volatile computer-readable storage medium storing a program of a spatio-temporal feature extraction method based on MRI images, and the spatio-temporal feature extraction based on MRI images can be executed by at least one processor, so that the at least one processor executes the steps of the spatio-temporal feature extraction method based on MRI images as described above.

[0109] Through the description of the above implementation manners, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation manner. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0110] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing specific implementation manners, or perform equivalent replacements for some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present application in other related technical fields shall be similarly within the scope of the patent protection of the present application.

Claims

1. A method for extracting spatiotemporal features based on MRI images, characterized in that: include: Perform preliminary feature extraction on the MRI image sequence to obtain the initial features of the MRI image; The initial features are weighted based on the temporal attention mechanism to highlight the features of key time steps, further optimize the feature representation, and obtain optimized features; Dynamic temporal convolution further extracts and refines the optimized features at different time and spatial scales by dynamically adjusting the convolution kernel to obtain the deep temporal and spatial characteristics of MRI images.

2. The method for extracting spatiotemporal features based on MRI images according to claim 1, characterized in that: The LSTM-GRU hybrid network is used to perform preliminary feature extraction on the MRI image sequence to obtain the initial features of the MRI image.

3. The method for extracting spatiotemporal features based on MRI images according to claim 2, characterized in that: Before the MRI image enters the LSTM-GRU hybrid network, the MRI image is preprocessed. The preprocessing includes: The grayscale values ​​of the MRI images are normalized to a preset range, and the size of the MRI images is adjusted so that the MRI images have the same spatial resolution and size.

4. The method for extracting spatiotemporal features based on MRI images according to claim 3, characterized in that: Before the MRI image enters the LSTM-GRU hybrid network, the MRI image is preprocessed, which also includes: The preprocessed MRI image is input into the first convolutional layer of the convolutional neural network; The convolution kernel slides on the image according to the set step size and performs convolution operation on each local area; each convolution kernel generates a corresponding feature map, and multiple convolution kernels operate in parallel to obtain multiple feature maps; Repeat the above convolution process, use the output of the previous convolution layer as the input of the next convolution layer, and gradually extract features of different levels of the image by stacking multiple convolution layers; After several convolutional layers and activation function layers, a pooling layer is inserted to divide the feature map into several non-overlapping local areas. The maximum value in each local area is taken as the result after pooling to obtain the downsampled feature map. The feature maps after multiple convolution, activation and pooling operations are fused, and the fused features are used as the output of the convolutional neural network.

5. The method for extracting spatiotemporal features based on MRI images according to claim 1, characterized in that: Based on the temporal attention mechanism, the initial features are weighted to highlight the features of the key time steps, further optimize the feature representation, and obtain optimized features, including: For the temporal attention mechanism, input the initial feature sequence X = [x1, x2, ..., x T ], where T is the number of time steps, x t is the feature vector of the tth time step, with dimension d; Map the input initialized feature sequence to three spaces: query Q, key K, and value V; Calculate the similarity between the query and the key to get the attention score; Convert the attention score into probability distribution and obtain the attention weight; Based on the calculated attention weights, the value vector is weighted summed to obtain the optimized features for each time step where v i is the value vector of the i-th time step; finally, the optimized feature sequence is obtained 6. The method for extracting spatiotemporal features based on MRI images according to claim 5, characterized in that: Convert the attention score into a probability distribution and obtain the attention weight, including: Convert the attention score into a probability distribution and normalize it using the softmax function: where α t,i is the attention weight of the t-th time step to the i-th time step, satisfying 7. A spatiotemporal feature extraction system based on MRI images, used to implement the spatiotemporal feature extraction method based on MRI images according to claims 1-6, characterized in that: include: An initial feature acquisition module is used to perform preliminary feature extraction on the MRI image sequence and obtain the initial features of the MRI image; The optimized feature acquisition module is used to weight the initial features based on the temporal attention mechanism, highlight the features of key time steps, further optimize the feature representation, and obtain optimized features; The deep spatiotemporal feature acquisition module is used for dynamic time convolution to further extract and refine the optimized features at different time and space scales by dynamically adjusting the convolution kernel to obtain the deep spatiotemporal features of MRI images.

8. An electronic device, characterized in that: include: one or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to perform the steps of the method for extracting spatiotemporal features based on MRI images as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the steps of the method for extracting spatiotemporal features based on MRI images as claimed in any one of claims 1 to 6.

Images