Intraoperative bridging cerebral vessel blood flow assessment system based on machine vision

Abstract

The application relates to the field of intracranial and extracranial vascular bypass, and particularly discloses a machine vision-based intraoperative bypass cerebral vascular blood flow evaluation system, which displays real-time registration based on intraoperative infrared thermal imaging data and preoperative cerebral perfusion images, that is, uses a deep neural network model as a feature extractor of image frames to respectively excavate spatial position distribution feature information about cerebral vessels in the intraoperative infrared thermal imaging data and the preoperative cerebral perfusion images and dynamic change feature information about blood flow in each image frame, and then displays the preoperative and intraoperative images by superimposed fusion through affine registration and elastic registration to dynamically monitor blood flow changes in the entire cerebral tissue vascular in the bypass operation in real time, and accurately judge the patency of blood flow in the bypass vascular. In this way, the surgical precision and safety can be improved, and the health of the people is effectively ensured.

Description

Technical Field

[0001] This invention relates to the field of intracranial and extracranial vascular bypass, and more specifically, to a machine vision-based intraoperative cerebral blood flow assessment system for bypass surgery. Background Technology

[0002] Intracranial and extracranial vascular bypass surgery is widely used in the treatment of intracranial diseases such as Moyamoya disease and complex intracranial aneurysms. In recent years, with changes in the living environment and the development of imaging technology, the detection rate of related diseases has been increasing year by year, and the number of cases receiving treatment has been increasing. Currently, nearly 100 Moyamoya disease and intracranial aneurysm bypass surgeries are performed annually at Ningbo First Hospital alone.

[0003] Intracranial and extracranial vascular bypass surgery is a primary treatment for cerebrovascular diseases, especially ischemic moyamoya disease or complex aneurysms. Accurately assessing the patency of the anastomosed vessels is crucial for surgical success. Currently, the main method for assessing vascular patency is fluorescence angiography; however, fluorescent contrast agents (such as indocyanine green) pose an allergy risk. Furthermore, due to metabolic reasons, the interval between repeated use of fluorescence angiography during the entire procedure must be greater than 15 minutes, but the image quality will still be significantly reduced due to residual contrast agent. In addition, the above methods can only observe blood flow velocity and direction, and cannot determine changes in brain function metabolism.

[0004] Infrared technology is widely used in rehabilitation medicine and microsurgery, and has also been used to detect brain temperature during neurosurgical bypass surgery. Hemodynamics provides reliable evidence for clinical diagnosis, but direct hemodynamic studies on patients are subject to many limitations. Animal experiments are difficult and costly. Microfluidic technology can construct microchannel networks with complex three-dimensional structures, and by quantitatively controlling the fluid inlet boundary conditions, key characteristics of blood flow can be accurately simulated. Domestic and international researchers, such as He Yong of Zhejiang University and Holger Gerhardtg of the Helmholtz Association Center for Molecular Medicine, have conducted extensive hemodynamic studies based on microfluidic technology. Domestic research institutions have also applied for a series of application patents for vascularized tissue structures based on microfluidic chips. However, due to the complex and variable distribution of the brain's vascular network and the lack of neural network algorithms for image processing of specific networks, the quality of intraoperative images of blood flow distribution and metabolic changes is often unsatisfactory.

[0005] Therefore, there is a need for a machine vision-based intraoperative cerebral bypass blood flow assessment system that can monitor changes in blood flow throughout the brain tissue during surgery, accurately assess intraoperative blood flow and vascular patency, thereby improving surgical precision and safety and effectively protecting public health. Summary of the Invention

[0006] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a machine vision-based intraoperative cerebral bypass blood flow assessment system. This system utilizes real-time registration and display of intraoperative infrared thermal imaging data and preoperative brain perfusion images. Specifically, a deep neural network model is used as a feature extractor for image frames to extract spatial distribution features of cerebral blood vessels from both the intraoperative infrared thermal imaging data and the preoperative brain perfusion images, as well as dynamic changes in blood flow within each image frame. Then, through affine and elastic registration, the preoperative and intraoperative images are superimposed and fused for display, dynamically showing the distribution of cortical blood flow during intraoperative cerebral bypass surgery and assessing whether the expected blood flow has been achieved. This allows for real-time monitoring of blood flow changes within the entire brain tissue's blood vessels, enabling accurate assessment of the patency of blood flow within the bypass vessels, thereby improving surgical precision and safety, and effectively protecting public health.

[0007] According to one aspect of this application, a machine vision-based intraoperative cerebral bypass blood flow assessment system is provided, comprising:

[0008] The data acquisition module is used to acquire preoperative brain perfusion images and intraoperative infrared thermal monitoring videos;

[0009] The sampling module is used to extract multiple image keyframes from the intraoperative infrared thermal monitoring video;

[0010] The noise reduction module is used to pass the multiple image keyframes through an image noise reduction device based on an autoencoder to obtain multiple noise-reduced image keyframes.

[0011] The cerebral vascular image feature extraction module is used to obtain multiple cerebral vascular feature maps by using a first convolutional neural network model with spatial attention mechanism for each of the multiple denoised keyframes of the image.

[0012] The semantic segmentation module is used to perform image semantic segmentation on the multiple cerebral blood vessel feature maps to obtain multiple cerebral blood vessel segmentation maps;

[0013] The brain perfusion image feature extraction module is used to process the preoperative brain perfusion image through a second convolutional neural network model, which serves as a feature extractor, to obtain a preoperative brain perfusion feature map.

[0014] The registration and fusion module is used to fuse each of the multiple cerebral blood vessel segmentation maps with the preoperative cerebral perfusion feature map based on coordinate space transformation to obtain multiple registration and fusion feature maps.

[0015] A dynamic feature extraction module is used to obtain a tracking feature map by using a third convolutional neural network model with a time attention mechanism to fuse the multiple registered feature maps; and

[0016] The evaluation result generation module is used to pass the tracking feature map through a classifier to obtain a classification result, which is used to indicate whether the intraoperative blood flow in the bypass cerebral vessels meets the predetermined requirements.

[0017] In the aforementioned machine vision-based intraoperative bypass cerebral blood flow assessment system, the sampling module includes: a first keyframe setting unit, used to set an initial image frame in the intraoperative infrared thermal monitoring video as a first image keyframe; a difference unit, used to calculate the positional pixel difference between other image frames in the intraoperative infrared thermal monitoring video along the time dimension and the first image keyframe to obtain a difference image frame; a statistical feature value calculation unit, used to calculate the statistical feature value of the pixel difference at all positions in the difference image frame; and a result determination unit, used to determine whether other image frames corresponding to the difference image frame are image keyframes based on a comparison between the statistical feature value and a predetermined threshold.

[0018] In the aforementioned machine vision-based intraoperative bypass cerebral blood flow assessment system, the statistical feature value calculation unit is further used to: calculate the mean and standard deviation of the pixel differences at all locations in the differential image frame as the statistical feature values.

[0019] In the aforementioned machine vision-based intraoperative bypass cerebral blood flow assessment system, the result determination unit is further configured to: calculate the weighted sum of the standard deviation and the mean to obtain a difference feature value; and, in response to the difference feature value being greater than or equal to the predetermined threshold, determine other image frames corresponding to the difference image frame as the image keyframes.

[0020] In the aforementioned machine vision-based intraoperative bypass cerebral blood flow assessment system, the noise reduction module is further configured to: use a depth-separable convolutional neural network to perform convolution on filters of different layers in its convolution operation in two spatial dimensions and channel dimensions to reduce noise on the multiple image keyframes based on three-dimensional block matching and filtering, so as to obtain multiple noise-reduced image keyframes.

[0021] In the aforementioned machine vision-based intraoperative bypass cerebral blood flow assessment system, the cerebral vascular image feature extraction module is further configured to: In the forward propagation of the first convolutional neural network model, each layer performs the following on the input data: convolution processing based on a two-dimensional convolutional kernel to generate a convolutional feature map; pooling processing on the convolutional feature map to generate a pooled feature map; activation processing on the pooled feature map to generate an activation feature map; global average pooling along the channel dimension on the activation feature map to obtain a spatial feature matrix; convolution and activation processing on the spatial feature matrix to generate a weight vector; and weighting each feature matrix of the activation feature map with the weight values ​​at each position in the weight vector to obtain a generated feature map; wherein the generated feature map output by the last layer of the first convolutional neural network model is the plurality of cerebral vascular feature maps.

[0022] In the aforementioned machine vision-based intraoperative bypass cerebral blood flow assessment system, the brain perfusion image feature extraction module is further configured to: perform convolution, pooling, and activation processing on the input data in the forward propagation of the layers of the second convolutional neural network model serving as the feature extractor, so as to generate the preoperative brain perfusion feature map by the last layer of the second convolutional neural network model serving as the feature extractor, wherein the input of the first layer of the second convolutional neural network model serving as the feature extractor is the preoperative brain perfusion image.

[0023] In the aforementioned machine vision-based intraoperative cerebral bypass blood flow assessment system, the dynamic feature extraction module includes: an adjacent frame extraction unit, used to extract adjacent first-frame registration fusion feature maps and second-frame registration fusion feature maps from the plurality of registration fusion feature maps; a first image local feature extraction unit, used to pass the first-frame registration fusion feature map and the second-frame registration fusion feature map through the first convolutional layer and the second convolutional layer of the third convolutional neural network model respectively to obtain a first-frame feature map corresponding to the first frame and a second-frame feature map corresponding to the second frame; a temporal attention unit, used to perform positional dot product between the first-frame feature map and the second-frame feature map and then pass them through the Softmax activation function to obtain a temporal attention map; a second image local feature extraction unit, used to pass the first frame through the third convolutional layer of the third convolutional neural network model to obtain a local feature map; and a temporal attention application unit, used to perform positional dot product between the local feature map and the temporal attention map to obtain the tracking feature map.

[0024] In the aforementioned machine vision-based intraoperative bypass cerebral blood flow assessment system, the time attention unit is further used to: calculate the time attention map by multiplying the first frame feature map and the second frame feature map by position using the following formula and then using the Softmax activation function.

[0025] The formula is as follows:

[0026] F A =Softmax[Cov(F t )⊙Cov(F t-1 )]

[0027] Among them, F A Represents the time attention map, F t-1 F represents the registration and fusion feature map of the second frame. t represents the first frame registration and fusion feature map, Cov(·) represents the convolution process of multiple convolutional layers on the image frame registration and fusion feature map, and ⊙ represents positional dot product.

[0028] In the aforementioned machine vision-based intraoperative cerebral bypass intravascular blood flow assessment system, the assessment result generation module is further configured to: process the tracking feature map using the classifier according to the following formula to generate a classification result, wherein the formula is: softmax{(W n B n ):…:(W1,B1)|Project(F)}, where Project(F) represents projecting the tracking feature map into a vector, W1 to W n Here are the weight matrices for each fully connected layer, B1 to B... n This represents the bias matrix of each fully connected layer.

[0029] Compared with existing technologies, the machine vision-based intraoperative cerebral bypass blood flow assessment system provided in this application utilizes real-time registration and display of intraoperative infrared thermal imaging data and preoperative brain perfusion images. Specifically, a deep neural network model is used as a feature extractor for image frames to extract spatial distribution features of cerebral blood vessels from both the intraoperative infrared thermal imaging data and the preoperative brain perfusion images, as well as dynamic changes in blood flow within each image frame. Then, through affine and elastic registration, the preoperative and intraoperative images are superimposed and fused for display, dynamically showing the distribution of cortical blood flow during intraoperative cerebral bypass surgery and assessing whether the expected blood flow has been achieved. This allows for real-time monitoring of blood flow changes within the entire brain tissue, enabling accurate assessment of the patency of blood flow within the bypass vessels, thereby improving surgical precision and safety, and effectively protecting public health. Attached Figure Description

[0030] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0031] Figure 1 This is a block diagram of a machine vision-based intraoperative bypass cerebral blood flow assessment system according to an embodiment of this application.

[0032] Figure 2 This is a block diagram of the sampling module in a machine vision-based intraoperative bypass cerebral blood flow assessment system according to an embodiment of this application.

[0033] Figure 3 This is a block diagram of the dynamic feature extraction module in a machine vision-based intraoperative bypass cerebral blood flow assessment system according to an embodiment of this application.

[0034] Figure 4 This is a schematic diagram of the architecture of a machine vision-based intraoperative bypass cerebral blood flow assessment system according to an embodiment of this application. Detailed Implementation

[0035] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0036] Scene Overview

[0037] As mentioned earlier, intracranial and extracranial vascular bypass surgery is widely used in the treatment of intracranial diseases such as Moyamoya disease and complex intracranial aneurysms. In recent years, with changes in the living environment and the development of imaging technology, the detection rate of related diseases has been increasing year by year, and the number of cases receiving treatment has been increasing. Currently, nearly 100 Moyamoya disease and intracranial aneurysm bypass surgeries are performed annually at Ningbo First Hospital alone.

[0038] Intracranial and extracranial vascular bypass surgery is a primary treatment for cerebrovascular diseases, especially ischemic moyamoya disease or complex aneurysms. Accurately assessing the patency of the anastomosed vessels is crucial for surgical success. Currently, the main method for assessing vascular patency is fluorescence angiography; however, fluorescent contrast agents (such as indocyanine green) pose an allergy risk. Furthermore, due to metabolic reasons, the interval between repeated use of fluorescence angiography during the entire procedure must be greater than 15 minutes, but the image quality will still be significantly reduced due to residual contrast agent. In addition, the above methods can only observe blood flow velocity and direction, and cannot determine changes in brain function metabolism.

[0039] Infrared technology is widely used in rehabilitation medicine and microsurgery, and has also been used to detect brain temperature during neurosurgical bypass surgery. Hemodynamics provides reliable evidence for clinical diagnosis, but direct hemodynamic studies on patients are subject to many limitations. Animal experiments are difficult and costly. Microfluidic technology can construct microchannel networks with complex three-dimensional structures, and by quantitatively controlling the fluid inlet boundary conditions, key characteristics of blood flow can be accurately simulated. Domestic and international researchers, such as He Yong of Zhejiang University and Holger Gerhardtg of the Helmholtz Association Center for Molecular Medicine, have conducted extensive hemodynamic studies based on microfluidic technology. Domestic research institutions have also applied for a series of application patents for vascularized tissue structures based on microfluidic chips. However, due to the complex and variable distribution of the brain's vascular network and the lack of neural network algorithms for image processing of specific networks, the quality of intraoperative images of blood flow distribution and metabolic changes is often unsatisfactory.

[0040] Therefore, there is a need for a machine vision-based intraoperative cerebral bypass blood flow assessment system that can monitor changes in blood flow throughout the brain tissue during surgery, accurately assess intraoperative blood flow and vascular patency, thereby improving surgical precision and safety and effectively protecting public health.

[0041] The Intelligent Medical Imaging Team at the Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, has long been engaged in research related to medical image analysis, surgical navigation, and mixed reality. In the field of medical image processing, they have conducted systematic and in-depth research on the theories and technologies of vascular structure segmentation and 3D modeling, image enhancement and display, medical image feature extraction, and multimodal image registration. They have participated in the transformation of related projects and obtained three national medical device registration certificates. In the past five years, they have published over 60 papers as first or corresponding author in authoritative international journals and academic conferences. Their first-author paper in IEEE-TMI has been cited up to 303 times (vascular segmentation algorithm), was named the "Most Popular Paper" of the year by IEEE-TMI in 2015, and was selected as an ESI highly cited paper (1%). Their paper in Medical Physics, published as corresponding author, was featured on the journal cover. Their paper in MICCAI, published as corresponding author, was selected as one of Springer Nature's "500+ Most Influential International Academic Papers by Chinese Authors" in 2020. Based on the team's accumulated expertise in automated medical image processing, the feasibility of this solution in intraoperative infrared thermal imaging and cerebral vascular hemodynamic analysis is guaranteed.

[0042] Based on this, in the technical solution of this application, the intraoperative infrared thermal imaging data and the preoperative brain perfusion image are registered and displayed in real time. Then, through affine registration and elastic registration, the preoperative and intraoperative images are superimposed and fused for display, so as to display the distribution of cerebral cortical blood flow in real time and dynamically, and assess whether the cerebral cortical blood flow has reached the expected level.

[0043] Specifically, considering that accurately determining the patency of blood flow within the bypass vessels during surgery relies on real-time segmentation of the cerebral vascular region via intraoperative infrared thermography, a cerebral vascular segmentation framework based on convolutional neural networks (CNNs) was first designed for real-time localization and segmentation of cerebral vessels using infrared thermography video data. A detailed cerebral vascular information aggregation module was then integrated into the framework to improve the segmentation accuracy. Finally, hemodynamic analysis techniques were applied to the segmented cerebral vascular region to accurately obtain blood flow parameters within the donor and recipient vessels.

[0044] Specifically, in the technical solution of this application, firstly, preoperative brain perfusion images and intraoperative infrared thermal monitoring videos are acquired using a camera. Then, regarding the intraoperative infrared thermal monitoring video, considering that it is a set of multiple dynamic image frames in the time dimension, the data volume of the image frames in the infrared thermal monitoring video is large, and there are dynamic correlations between multiple image frames. Therefore, in order to accurately assess intraoperative blood flow in the bypass cerebral vessels based on the implicit dynamic correlation feature information between each image frame, multiple keyframes are further extracted from the intraoperative infrared thermal monitoring video for keyframe image feature extraction. Accordingly, in a specific example of this application, the difference frame method can be used to extract multiple keyframes from the intraoperative infrared thermal monitoring video.

[0045] Furthermore, in order to filter out interference from irrelevant factors such as the surrounding environment on the judgment of blood flow in each image frame of the intraoperative infrared thermal monitoring video, the multiple image keyframes are respectively processed by an image denoising device based on an autoencoder to obtain multiple denoised image keyframes, so as to focus on the state characteristics of blood flow in the intraoperative bypass cerebral vessels in each image frame. In this embodiment, a deep separable convolutional neural network based on an autoencoder image denoising device can be used for denoising processing. In particular, the filters of different layers of the deep separable convolutional neural network are used to perform convolution in two spatial dimensions and channel dimensions respectively in its convolution operation to denoise the multiple image keyframes based on three-dimensional block matching and filtering to obtain multiple denoised image keyframes.

[0046] Then, in order to focus on spatial location information in the deep dynamic feature mining of intracerebral blood flow in each key frame of the intraoperative infrared thermal monitoring video, a first convolutional neural network model with a spatial attention mechanism, which has excellent performance in extracting latent features of spatial distribution, is used to process each of the multiple denoised key frames to extract high-dimensional features of the spatial location distribution of the cerebral blood vessels in each denoised key frame, thereby obtaining multiple cerebral blood vessel feature maps. Then, image semantic segmentation is performed on each of the multiple cerebral blood vessel feature maps to divide them into different cerebral blood vessel spatial location distribution region feature maps in the high-dimensional feature space, thereby obtaining multiple cerebral blood vessel segmentation maps.

[0047] Accordingly, for the preoperative brain perfusion images, a second convolutional neural network model, which has excellent performance in extracting local implicit correlation features, is used as a feature extractor to perform deep feature mining on the preoperative brain perfusion images, so as to extract the feature distribution representation of local features in the preoperative brain perfusion images in a high-dimensional feature space, thereby obtaining a preoperative brain perfusion feature map.

[0048] Therefore, to improve the accuracy of feature fusion and thus enhance the accuracy of assessing intraoperative blood flow in the bypass cerebral vessels, the technical solution of this application fuses each of the multiple cerebral vessel segmentation maps with the preoperative cerebral perfusion feature map based on coordinate space transformation to obtain multiple registration-fused feature maps. Furthermore, it should be understood that both the preoperative cerebral perfusion image and the intraoperative infrared thermal monitoring video exhibit dynamic feature correlations in the time dimension, i.e., dynamic change features. Therefore, to dynamically display the distribution of bleeding flow in real time when assessing blood flow, a third convolutional neural network model with a time attention mechanism is used to extract features from the multiple registration-fused feature maps. This extracts the implicit dynamic changes in blood flow distribution across each image frame in the time dimension, thereby obtaining a tracking feature map.

[0049] Finally, a classifier is used to classify the tracking feature map to obtain a classification result indicating whether the blood flow in the bypass vessels during surgery meets predetermined requirements. This allows for real-time dynamic display of cortical blood flow distribution during cerebral vascular bypass surgery, assessment of whether cortical blood flow meets expectations, and real-time monitoring of blood flow changes throughout the brain tissue vessels. This enables accurate judgment of the patency of blood flow in the bypass vessels, improving surgical precision and safety, and effectively protecting public health.

[0050] Based on this, this application proposes a machine vision-based intraoperative cerebral bypass blood flow assessment system, comprising: a data acquisition module for acquiring preoperative cerebral perfusion images and intraoperative infrared thermal monitoring videos; a sampling module for extracting multiple image keyframes from the intraoperative infrared thermal monitoring videos; a noise reduction module for passing the multiple image keyframes through an image noise reduction device based on an autoencoder to obtain multiple noise-reduced image keyframes; a cerebral vascular image feature extraction module for extracting multiple cerebral vascular feature maps by using a first convolutional neural network model with a spatial attention mechanism for each of the multiple noise-reduced image keyframes; and a semantic segmentation module for performing image semantic segmentation on the multiple cerebral vascular feature maps to obtain... The system comprises: a preoperative brain perfusion image feature extraction module, which extracts the preoperative brain perfusion image using a second convolutional neural network model as a feature extractor to obtain a preoperative brain perfusion feature map; a registration and fusion module, which fuses each preoperative brain perfusion feature map with the preoperative brain perfusion feature map based on coordinate space transformation to obtain multiple registration and fusion feature maps; a dynamic feature extraction module, which extracts the multiple registration and fusion feature maps using a third convolutional neural network model with a time attention mechanism to obtain a tracking feature map; and an evaluation result generation module, which passes the tracking feature map through a classifier to obtain a classification result, wherein the classification result indicates whether the intraoperative blood flow in the bypass cerebral vessels meets predetermined requirements.

[0051] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0052] Exemplary System

[0053] Figure 1 The illustration shows a block diagram of a machine vision-based intraoperative cerebral bypass blood flow assessment system according to an embodiment of this application. Figure 1As shown, the intraoperative cerebral bypass blood flow assessment system 200 based on machine vision according to an embodiment of this application includes: a data acquisition module 210 for acquiring preoperative brain perfusion images and intraoperative infrared thermal monitoring videos; a sampling module 220 for extracting multiple image keyframes from the intraoperative infrared thermal monitoring videos; a noise reduction module 230 for passing the multiple image keyframes through an image noise reduction device based on an autoencoder to obtain multiple noise-reduced image keyframes; a cerebral vascular image feature extraction module 240 for obtaining multiple cerebral vascular feature maps by using a first convolutional neural network model with a spatial attention mechanism for each of the multiple noise-reduced image keyframes; and a semantic segmentation module 250 for performing image semantic segmentation on the multiple cerebral vascular feature maps. The procedure involves: obtaining multiple cerebral vascular segmentation maps; a brain perfusion image feature extraction module 260, which uses a second convolutional neural network model as a feature extractor to obtain a preoperative brain perfusion feature map; a registration and fusion module 270, which fuses each cerebral vascular segmentation map with the preoperative brain perfusion feature map based on coordinate space transformation to obtain multiple registration and fusion feature maps; a dynamic feature extraction module 280, which uses a third convolutional neural network model with a time attention mechanism to obtain a tracking feature map from the multiple registration and fusion feature maps; and an evaluation result generation module 290, which uses a classifier to obtain a classification result from the tracking feature map, the classification result indicating whether the intraoperative blood flow in the bypassed cerebral vessels meets predetermined requirements.

[0054] Specifically, in this embodiment, the data acquisition module 210 and the sampling module 220 are used to acquire preoperative brain perfusion images and intraoperative infrared thermal monitoring videos, and extract multiple image keyframes from the intraoperative infrared thermal monitoring videos. As mentioned above, in the technical solution of this application, it is desirable to perform real-time registration and display based on intraoperative infrared thermal imaging data and preoperative brain perfusion images. That is, through affine registration and elastic registration, the preoperative and intraoperative images are superimposed and fused for display, so as to dynamically display the distribution of cerebral cortical blood flow during intraoperative cerebral vascular bypass surgery and assess whether the cerebral cortical blood flow has reached the expected level.

[0055] Specifically, in the technical solution of this application, firstly, preoperative brain perfusion images and intraoperative infrared thermal monitoring videos are acquired using a camera. Then, regarding the intraoperative infrared thermal monitoring video, considering that it is a set of multiple dynamic image frames in the time dimension, the data volume of the image frames in the infrared thermal monitoring video is large, and there are dynamic correlations between multiple image frames. Therefore, in order to accurately assess intraoperative blood flow in the bypass cerebral vessels based on the implicit dynamic correlation feature information between each image frame, multiple keyframes are further extracted from the intraoperative infrared thermal monitoring video for keyframe image feature extraction. Accordingly, in a specific example of this application, the difference frame method can be used to extract multiple keyframes from the intraoperative infrared thermal monitoring video.

[0056] More specifically, in this embodiment, the sampling module includes: First, setting an initial image frame in the intraoperative infrared thermal monitoring video as a first image keyframe. Next, calculating the positional pixel differences between other image frames along the time dimension in the intraoperative infrared thermal monitoring video and the first image keyframe to obtain a differential image frame. Then, calculating statistical feature values ​​of the pixel differences at all positions in the differential image frame. Accordingly, in a specific example, the mean and standard deviation of the pixel differences at all positions in the differential image frame can be calculated as the statistical feature values. Finally, based on a comparison between the statistical feature values ​​and a predetermined threshold, determining whether other image frames corresponding to the differential image frame are image keyframes. Accordingly, in a specific example, calculating a weighted sum of the standard deviation and the mean to obtain a difference feature value; and, in response to the difference feature value being greater than or equal to the predetermined threshold, determining that other image frames corresponding to the differential image frame are the image keyframes.

[0057] Figure 2 The diagram illustrates a block diagram of the sampling module in a machine vision-based intraoperative cerebral bypass blood flow assessment system according to an embodiment of this application. Figure 2 As shown, the sampling module 220 includes: a first keyframe setting unit 221, used to set the initial image frame in the intraoperative infrared thermal monitoring video as the first image keyframe; a difference unit 222, used to calculate the positional pixel difference between other image frames in the intraoperative infrared thermal monitoring video along the time dimension and the first image keyframe to obtain a difference image frame; a statistical feature value calculation unit 223, used to calculate the statistical feature value of the pixel difference at all positions in the difference image frame; and a result determination unit 224, used to determine whether other image frames corresponding to the difference image frame are image keyframes based on the comparison between the statistical feature value and a predetermined threshold.

[0058] Specifically, in this embodiment, the noise reduction module 230 is used to pass the plurality of image keyframes through an autoencoder-based image noise reduction device to obtain a plurality of noise-reduced image keyframes. That is, in the technical solution of this application, in order to filter out interference from irrelevant factors such as the surrounding environment on the judgment of blood flow in each image frame of the intraoperative infrared thermal monitoring video, the plurality of image keyframes are further passed through an autoencoder-based image noise reduction device to obtain a plurality of noise-reduced image keyframes, so as to focus on the state characteristics of blood flow within the intraoperative bypass cerebral vessels in each image frame.

[0059] Accordingly, in a specific example, a deep separable convolutional neural network based on an autoencoder image denoising device can be used for denoising. In particular, the filters of different layers of the deep separable convolutional neural network are used to convolve in two spatial dimensions and channel dimensions respectively in its convolution operation to denoise the multiple image keyframes based on three-dimensional block matching and filtering to obtain multiple denoised image keyframes.

[0060] Specifically, in this embodiment, the cerebral blood vessel image feature extraction module 240 and the semantic segmentation module 250 are used to obtain multiple cerebral blood vessel feature maps by using a first convolutional neural network model with spatial attention mechanism for each of the multiple denoised image keyframes, and to perform image semantic segmentation on the multiple cerebral blood vessel feature maps to obtain multiple cerebral blood vessel segmentation maps. It should be understood that, in order to focus on spatial location information in the deep dynamic feature mining of intracerebral blood flow in each keyframe of the intraoperative infrared thermal monitoring video, the technical solution of this application further uses a first convolutional neural network model with spatial attention mechanism, which has excellent performance in extracting hidden features of spatial distribution, to process each of the multiple denoised image keyframes to extract high-dimensional features of the spatial location distribution of the cerebral blood vessels in each denoised image keyframe, thereby obtaining multiple cerebral blood vessel feature maps. Then, image semantic segmentation is performed on the multiple cerebral blood vessel feature maps to divide them into different cerebral blood vessel spatial location distribution region feature maps in the high-dimensional feature space, thereby obtaining multiple cerebral blood vessel segmentation maps.

[0061] More specifically, in this embodiment, the cerebral vascular image feature extraction module is further configured to: perform the following operations on the input data during the forward propagation of the first convolutional neural network model: perform convolution processing on the input data based on a two-dimensional convolutional kernel to generate a convolutional feature map; perform pooling processing on the convolutional feature map to generate a pooled feature map; perform activation processing on the pooled feature map to generate an activation feature map; perform global average pooling along the channel dimension on the activation feature map to obtain a spatial feature matrix; perform convolution processing and activation processing on the spatial feature matrix to generate a weight vector; and weight each feature matrix of the activation feature map with the weight values ​​at each position in the weight vector to obtain a generated feature map; wherein the generated feature map output by the last layer of the first convolutional neural network model is the plurality of cerebral vascular feature maps.

[0062] Specifically, in this embodiment, the brain perfusion image feature extraction module 260 is used to process the preoperative brain perfusion image through a second convolutional neural network model, which serves as a feature extractor, to obtain a preoperative brain perfusion feature map. That is, in the technical solution of this application, for the preoperative brain perfusion image, a second convolutional neural network model, which has excellent performance in extracting local implicit correlation features, is further used to perform deep feature mining on the preoperative brain perfusion image, so as to extract the feature distribution representation of local features in the preoperative brain perfusion image in a high-dimensional feature space, thereby obtaining a preoperative brain perfusion feature map.

[0063] More specifically, in a specific example, each layer of the second convolutional neural network model, which serves as a feature extractor, performs convolution, pooling, and activation processing on the input data during the forward pass of the layer to generate the preoperative brain perfusion feature map by the last layer of the second convolutional neural network model, which serves as a feature extractor, wherein the input of the first layer of the second convolutional neural network model, which serves as a feature extractor, is the preoperative brain perfusion image.

[0064] Specifically, in this embodiment, the registration fusion module 270 and the dynamic feature extraction module 280 are used to fuse each cerebral blood vessel segmentation map in the plurality of cerebral blood vessel segmentation maps with the preoperative cerebral perfusion feature map based on coordinate space transformation to obtain a plurality of registration fusion feature maps, and to obtain a tracking feature map by using a third convolutional neural network model with a time attention mechanism. It should be understood that, in order to improve the accuracy of feature fusion and thus improve the accuracy of assessing intraoperative blood flow in the bypass cerebral vessels, the technical solution of this application further fuses each cerebral blood vessel segmentation map in the plurality of cerebral blood vessel segmentation maps with the preoperative cerebral perfusion feature map based on coordinate space transformation to obtain a plurality of registration fusion feature maps. Furthermore, considering that both the preoperative brain perfusion images and the intraoperative infrared thermal monitoring videos have dynamic feature associations in the time dimension, that is, dynamic change features, in order to dynamically display the distribution of bleeding flow in real time when assessing blood flow, a third convolutional neural network model with a time attention mechanism is used to extract features from the multiple registration and fusion feature maps, so as to extract the implicit features of dynamic changes in blood flow distribution in the time dimension of each image frame, thereby obtaining a tracking feature map.

[0065] More specifically, in this embodiment, the dynamic feature extraction module includes: first, extracting adjacent first-frame registration fusion feature maps and second-frame registration fusion feature maps from the plurality of registration fusion feature maps. Next, passing the first-frame registration fusion feature map and the second-frame registration fusion feature map through the first and second convolutional layers of the third convolutional neural network model to obtain a first-frame feature map corresponding to the first frame and a second-frame feature map corresponding to the second frame. Then, performing a positional dot product between the first-frame feature map and the second-frame feature map and applying a Softmax activation function to obtain a temporal attention map. Accordingly, in a specific example, the first-frame feature map and the second-frame feature map are performed a positional dot product using the following formula, and then the Softmax activation function is applied to calculate the temporal attention map.

[0066] The formula is as follows:

[0067] F A =Softmax[Cov(F t )⊙Cov(F t-1 )]

[0068] Among them, F A Represents the time attention map, F t-1 F represents the registration and fusion feature map of the second frame. tThe first frame registration fusion feature map is represented by Cov(·), which indicates that the image frame registration fusion feature map is processed by multiple convolutional layers, and ⊙ indicates positional dot product. Next, the first frame is passed through the third convolutional layer of the third convolutional neural network model to obtain a local feature map. Finally, the local feature map is multiplied by the temporal attention map to obtain the tracking feature map.

[0069] Figure 3 The diagram illustrates a block diagram of the dynamic feature extraction module in a machine vision-based intraoperative bypass cerebral blood flow assessment system according to an embodiment of this application. Figure 3 As shown, the dynamic feature extraction module 280 includes: an adjacent frame extraction unit 281, used to extract adjacent first frame registration fusion feature maps and second frame registration fusion feature maps from the plurality of registration fusion feature maps; a first image local feature extraction unit 282, used to pass the first frame registration fusion feature map and the second frame registration fusion feature map through the first convolutional layer and the second convolutional layer of the third convolutional neural network model respectively to obtain a first frame feature map corresponding to the first frame and a second frame feature map corresponding to the second frame; a temporal attention unit 283, used to perform positional dot product between the first frame feature map and the second frame feature map and then pass them through the Softmax activation function to obtain a temporal attention map; a second image local feature extraction unit 284, used to pass the first frame through the third convolutional layer of the third convolutional neural network model to obtain a local feature map; and a temporal attention application unit 285, used to perform positional dot product between the local feature map and the temporal attention map to obtain the tracking feature map.

[0070] Specifically, in this embodiment, the evaluation result generation module 290 is used to process the tracking feature map through a classifier to obtain a classification result, which is used to indicate whether the blood flow in the bypass cerebral vessels during surgery meets predetermined requirements. That is, in the technical solution of this application, a classifier is used to classify the tracking feature map to obtain a classification result indicating whether the blood flow in the bypass cerebral vessels during surgery meets predetermined requirements. This allows for real-time dynamic display of the cerebral cortex blood flow distribution during intraoperative cerebral vascular bypass surgery, assessment of whether the cerebral cortex blood flow meets expectations, and real-time monitoring of blood flow changes within the entire brain tissue vessels, thereby accurately determining the patency of blood flow in the bypass vessels. Accordingly, in a specific example, the classifier is used to process the tracking feature map using the following formula to generate a classification result, where the formula is: softmax{(W n B n):…:(W1,B1)|Project(F)}, where Project(F) represents projecting the tracking feature map into a vector, W1 to W n Here are the weight matrices for each fully connected layer, B1 to B... n This represents the bias matrix of each fully connected layer.

[0071] Figure 4 The illustration shows a schematic diagram of the architecture of a machine vision-based intraoperative bypass cerebral blood flow assessment system according to an embodiment of this application. Figure 4 As shown, in the network architecture of the machine vision-based intraoperative cerebral bypass blood flow assessment system, firstly, the intraoperative infrared thermal monitoring video (e.g., as shown) is obtained... Figure 4 The IN1 shown extracts multiple image keyframes (e.g., as shown in the image). Figure 4 As shown in P1); then, the plurality of image keyframes are respectively passed through an image denoising device based on an autoencoder (e.g., such as P1); Figure 4 The INR shown is used to obtain multiple denoised image keyframes (e.g., as shown in the figure). Figure 4 As shown in P2); then, each of the plurality of denoised keyframes is processed by a first convolutional neural network model with a spatial attention mechanism (e.g., such as P2); Figure 4 The CNN1 shown is used to obtain multiple cerebral vascular feature maps (e.g., such as...). Figure 4 As shown in F1); then, image semantic segmentation is performed on the multiple cerebral vascular feature maps to obtain multiple cerebral vascular segmentation maps (e.g., as shown in F1); Figure 4 (as shown in F); then, the obtained preoperative brain perfusion images (e.g., as shown in F) are... Figure 4 The IN2 shown is processed by a second convolutional neural network model (e.g., as shown) as a feature extractor. Figure 4 The CNN2 shown is used to obtain preoperative brain perfusion feature maps (e.g., as shown in the image). Figure 4 (as shown in F2); then, based on coordinate space transformation, each of the multiple cerebral vascular segmentation maps is fused with the preoperative cerebral perfusion feature map to obtain multiple registration fusion feature maps (e.g., such as F2); Figure 4 The multiple registration fusion feature maps are then processed using a third convolutional neural network model with a time attention mechanism (e.g., as shown in the FR); Figure 4 The CNN3 shown is used to obtain tracking feature maps (e.g., such as...). Figure 4 (as shown in the FT); finally, the tracking feature map is passed through a classifier (e.g., such as...) Figure 4 The classifier shown is used to obtain the classification result (e.g., as shown in the image). Figure 4The classification result (as shown in the CR) is used to indicate whether the intraoperative blood flow in the bypass cerebral vessels meets the predetermined requirements.

[0072] In summary, the machine vision-based intraoperative cerebral bypass blood flow assessment system 200 described in this application embodiment is explained. It achieves real-time registration and display of intraoperative infrared thermal imaging data and preoperative brain perfusion images. Specifically, a deep neural network model is used as a feature extractor for image frames to extract spatial distribution features of cerebral blood vessels from the intraoperative infrared thermal imaging data and the preoperative brain perfusion images, as well as dynamic changes in blood flow within each image frame. Then, through affine registration and elastic registration, the preoperative and intraoperative images are superimposed and fused for display, dynamically showing the distribution of cortical blood flow during intraoperative cerebral bypass surgery and assessing whether the expected blood flow has been achieved. This allows for real-time monitoring of blood flow changes within the entire brain tissue's blood vessels, enabling accurate assessment of the patency of blood flow within the bypass vessels, thereby improving surgical precision and safety and effectively protecting public health.

[0073] As described above, the machine vision-based intraoperative bypass cerebral blood flow assessment system 200 according to embodiments of this application can be implemented in various terminal devices, such as servers for machine vision-based intraoperative bypass cerebral blood flow assessment algorithms. In one example, the machine vision-based intraoperative bypass cerebral blood flow assessment system 200 according to embodiments of this application can be integrated into a terminal device as a software module and / or hardware module. For example, the machine vision-based intraoperative bypass cerebral blood flow assessment system 200 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the machine vision-based intraoperative bypass cerebral blood flow assessment system 200 can also be one of many hardware modules of the terminal device.

[0074] Alternatively, in another example, the machine vision-based intraoperative bypass intravascular blood flow assessment system 200 and the terminal device can also be separate devices, and the machine vision-based intraoperative bypass intravascular blood flow assessment system 200 can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0075] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0076] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0077] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0078] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0079] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A machine vision-based intraoperative cerebral bypass graft blood flow assessment system, characterized in that, include: The data acquisition module is used to acquire preoperative brain perfusion images and intraoperative infrared thermal monitoring videos, wherein the intraoperative infrared thermal monitoring videos capture the thermal dynamic changes of cerebral blood flow in real time. The sampling module extracts multiple key frames from the intraoperative infrared thermal monitoring video using the difference frame method. The key frames focus on the dynamic change features of the cerebral vascular region. The noise reduction module is used to pass the multiple image keyframes through an image noise reduction device based on an autoencoder to obtain multiple noise-reduced image keyframes. The image noise reduction device uses a depth-separable convolutional neural network to perform convolution processing in the spatial dimension and the channel dimension, and achieves targeted noise reduction based on three-dimensional block matching and filtering. The cerebral vascular image feature extraction module is used to obtain multiple cerebral vascular feature maps by using a first convolutional neural network model with spatial attention mechanism for each of the multiple denoised keyframes of the image. The semantic segmentation module is used to perform image semantic segmentation on the multiple cerebral blood vessel feature maps to obtain multiple cerebral blood vessel segmentation maps; The brain perfusion image feature extraction module is used to process the preoperative brain perfusion image through a second convolutional neural network model, which serves as a feature extractor, to obtain a preoperative brain perfusion feature map. The registration and fusion module is used to fuse each of the multiple cerebral blood vessel segmentation maps with the preoperative cerebral perfusion feature map based on coordinate space transformation to obtain multiple registration and fusion feature maps. A dynamic feature extraction module is used to obtain a tracking feature map by using a third convolutional neural network model with a time attention mechanism to fuse the multiple registered feature maps. The time attention mechanism focuses on the dynamic changes in blood flow over time through feature multiplication between adjacent frames and softmax activation. The dynamic feature extraction module includes: The adjacent frame extraction unit is used to extract adjacent first frame registration fusion feature maps and second frame registration fusion feature maps from the plurality of registration fusion feature maps; The first image local feature extraction unit is used to pass the first frame registration fusion feature map and the second frame registration fusion feature map through the first convolutional layer and the second convolutional layer of the third convolutional neural network model to obtain a first frame feature map corresponding to the first frame registration fusion feature map and a second frame feature map corresponding to the second frame registration fusion feature map. The temporal attention unit is used to perform positional dot product between the first frame feature map and the second frame feature map, and then pass the Softmax activation function to obtain the temporal attention map. The second image local feature extraction unit is used to pass the first frame registration and fusion feature map through the third convolutional layer of the third convolutional neural network model to obtain a local feature map; A temporal attention application unit is configured to perform a position-based dot product between the local feature map and the temporal attention map to obtain the tracking feature map; and The evaluation result generation module is used to pass the tracking feature map through a classifier to obtain a classification result, which is used to indicate whether the blood flow in the bypass cerebral vessels during the operation meets the predetermined requirements. The temporal attention unit is further configured to: multiply the first frame feature map and the second frame feature map by position using the following formula and then calculate the temporal attention map using the Softmax activation function; The formula is as follows: Among them, F A Represents the time attention map, F t-1 F represents the feature map of the second frame. t Let represent the first frame feature map, Cov(·) represents the convolution process of the image frame feature map with multiple convolutional layers, and ⊙ represents the dot product by position.

2. The intraoperative cerebral bypass blood flow assessment system based on machine vision according to claim 1, characterized in that, The sampling module includes: The first keyframe setting unit is used to set the initial image frame in the intraoperative infrared thermal monitoring video as the first image keyframe. The differential unit is used to calculate the positional pixel difference between other image frames along the time dimension in the intraoperative infrared thermal monitoring video and the first image key frame to obtain the differential image frame; A statistical feature value calculation unit is used to calculate the statistical feature values ​​of the pixel differences at all locations in the differential image frame; and The result determination unit is used to determine whether other image frames corresponding to the differential image frame are image keyframes based on the comparison between the statistical feature value and a predetermined threshold.

3. The intraoperative cerebral bypass blood flow assessment system based on machine vision according to claim 2, characterized in that, The statistical feature value calculation unit is further configured to: calculate the mean and standard deviation of the pixel differences at all positions in the differential image frame as the statistical feature values.

4. The intraoperative cerebral bypass blood flow assessment system based on machine vision according to claim 3, characterized in that, The result determination unit is further configured to: calculate the weighted sum of the standard deviation and the mean to obtain a difference feature value; and, in response to the difference feature value being greater than or equal to the predetermined threshold, determine other image frames corresponding to the difference image frame as the image keyframe.

5. The intraoperative cerebral bypass blood flow assessment system based on machine vision according to claim 1, characterized in that, The noise reduction module is further configured to: use a depth-separable convolutional neural network to perform convolution on filters of different layers in its convolution operation in two spatial dimensions and channel dimensions respectively to perform noise reduction on the multiple image keyframes based on three-dimensional block matching and filtering, so as to obtain multiple noise-reduced image keyframes.

6. The intraoperative cerebral bypass blood flow assessment system based on machine vision according to claim 1, characterized in that, The cerebral vascular image feature extraction module is further used for: each layer of the first convolutional neural network model performing the following operations on the input data during the forward propagation of the layer: The input data is subjected to convolution processing based on two-dimensional convolution kernels to generate a convolutional feature map; The convolutional feature map is subjected to pooling to generate a pooled feature map; The pooled feature map is activated to generate an activated feature map; The activated feature map is subjected to global average pooling along the channel dimension to obtain a spatial feature matrix; The spatial feature matrix is ​​subjected to convolution and activation processing to generate a weight vector; and The feature matrices of the activated feature map are weighted by the weight values ​​at each position in the weight vector to obtain the generated feature map. The generated feature map output from the last layer of the first convolutional neural network model is the plurality of cerebral vascular feature maps.

7. The intraoperative cerebral bypass blood flow assessment system based on machine vision according to claim 1, characterized in that, The brain perfusion image feature extraction module is further configured to: perform convolution, pooling and activation processing on the input data in the forward propagation of the layers of the second convolutional neural network model that serves as the feature extractor, so as to generate the preoperative brain perfusion feature map by the last layer of the second convolutional neural network model that serves as the feature extractor, wherein the input of the first layer of the second convolutional neural network model that serves as the feature extractor is the preoperative brain perfusion image.

8. A machine vision-based intraoperative cerebral bypass graft blood flow assessment system, characterized in that, include: The data acquisition module is used to acquire preoperative brain perfusion images and intraoperative infrared thermal monitoring videos; The sampling module is used to extract multiple image keyframes from the intraoperative infrared thermal monitoring video; The noise reduction module is used to pass the multiple image keyframes through an image noise reduction device based on an autoencoder to obtain multiple noise-reduced image keyframes. The image noise reduction device uses depth-separable convolution and combines it with three-dimensional block matching filtering to complete the noise reduction. The cerebral vascular image feature extraction module is used to obtain multiple cerebral vascular feature maps by using a first convolutional neural network model with spatial attention mechanism for each of the multiple denoised key frames of the image. The first convolutional neural network sequentially performs convolution, pooling, activation, and global average pooling to generate spatial weights and then weights them to obtain the cerebral vascular feature maps. The semantic segmentation module is used to perform image semantic segmentation on the multiple cerebral blood vessel feature maps to obtain multiple cerebral blood vessel segmentation maps; The brain perfusion image feature extraction module is used to process the preoperative brain perfusion image through a second convolutional neural network model, which serves as a feature extractor, to obtain a preoperative brain perfusion feature map. The second convolutional neural network sequentially performs convolution, pooling, and activation to output features. The registration and fusion module is used to fuse each of the multiple cerebral blood vessel segmentation maps with the preoperative cerebral perfusion feature map based on coordinate space transformation to obtain multiple registration and fusion feature maps. The dynamic feature extraction module is used to obtain a tracking feature map by using a third convolutional neural network model with a time attention mechanism to combine the multiple registered and fused feature maps. The dynamic feature extraction module includes a temporal attention unit, an adjacent frame extraction unit, and a first image local feature extraction unit. The adjacent frame extraction unit is used to extract adjacent first frame registration fusion feature maps and second frame registration fusion feature maps from the plurality of registration fusion feature maps. The first image local feature extraction unit is used to pass the first frame registration fusion feature map and the second frame registration fusion feature map through the first convolutional layer and the second convolutional layer of the third convolutional neural network model, respectively, to obtain a first frame feature map corresponding to the first frame registration fusion feature map and a second frame feature map corresponding to the second frame registration fusion feature map. The temporal attention unit is further configured to: multiply the first frame feature map and the second frame feature map by position using the following formula, and then calculate the temporal attention map using the Softmax activation function; The formula is as follows: Among them, F A Represents the time attention map, F t-1 F represents the feature map of the second frame. t The first frame feature map is represented by Cov(·), which indicates convolution processing of the image frame feature map using multiple convolutional layers, and ⊙ indicates positional dot product; and The evaluation result generation module is used to pass the tracking feature map through a classifier to obtain a classification result. The classification result is used to indicate whether the intraoperative blood flow in the bypass cerebral vessels meets predetermined requirements. The evaluation result generation module is further used to: process the tracking feature map using the classifier with the following formula to generate a classification result, wherein... The formula is: softmax{(W n B n ):…:(W1, B1)|Project(F)}, where Project(F) represents projecting the tracking feature map into a vector, W1 to W n Here are the weight matrices for each fully connected layer, B1 to B... n This represents the bias matrix of each fully connected layer.

9. The machine vision-based intraoperative cerebral bypass blood flow assessment system according to claim 8, characterized in that, The sampling module includes: The first keyframe setting unit is used to set the initial image frame in the intraoperative infrared thermal monitoring video as the first image keyframe. The differential unit is used to calculate the positional pixel difference between other image frames along the time dimension in the intraoperative infrared thermal monitoring video and the first image key frame to obtain the differential image frame; A statistical feature value calculation unit is used to calculate the statistical feature values ​​of the pixel differences at all locations in the differential image frame; and The result determination unit is used to determine whether other image frames corresponding to the differential image frame are image keyframes based on the comparison between the statistical feature value and a predetermined threshold.

10. The machine vision-based intraoperative cerebral bypass blood flow assessment system according to claim 9, characterized in that, The statistical feature value calculation unit is further configured to: calculate the mean and standard deviation of the pixel differences at all positions in the differential image frame as the statistical feature values.

11. The machine vision-based intraoperative cerebral bypass blood flow assessment system according to claim 10, characterized in that, The result determination unit is further configured to: calculate the weighted sum of the standard deviation and the mean to obtain a difference feature value; and, in response to the difference feature value being greater than or equal to the predetermined threshold, determine other image frames corresponding to the difference image frame as the image keyframe.

12. The machine vision-based intraoperative cerebral bypass blood flow assessment system according to claim 9, characterized in that, The noise reduction module is further configured to: use a depth-separable convolutional neural network to perform convolution on filters of different layers in its convolution operation in two spatial dimensions and channel dimensions respectively to perform noise reduction on the multiple image keyframes based on three-dimensional block matching and filtering, so as to obtain multiple noise-reduced image keyframes.

13. The machine vision-based intraoperative cerebral bypass blood flow assessment system according to claim 9, characterized in that, The brain perfusion image feature extraction module is further configured to: perform convolution, pooling and activation processing on the input data in the forward propagation of the layers of the second convolutional neural network model that serves as the feature extractor, so as to generate the preoperative brain perfusion feature map by the last layer of the second convolutional neural network model that serves as the feature extractor, wherein the input of the first layer of the second convolutional neural network model that serves as the feature extractor is the preoperative brain perfusion image.

Images