Method, device and equipment for assisting stroke assessment and medium

Abstract

The invention relates to the technical field of intelligent medical assistance, in particular to a method, device and equipment for assisting stroke assessment and a medium, and the method comprises the steps: receiving a brain image, clinical vital sign data, pre-hospital first-aid records and historical health data; the method comprises the following steps of: extracting deep semantic features of a brain image by adopting a convolutional neural network and a Swinin-Transform hybrid encoder; a gradient boosting tree feature encoder is adopted to extract nonlinear correlation features among the clinical vital sign data, the pre-hospital emergency records and the historical health data; based on the deep semantic features and the nonlinear correlation features, adopting residual connection and a multi-head attention mechanism to obtain complementary fusion features; determining the stroke type based on the fusion features; and when ischemic stroke is determined, the volume of a core infarction region is measured and calculated, so that the assessment time of the emergency stroke is shortened.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical assistance technology, and in particular to a device for assisting in stroke assessment. Background Technology

[0002] With the rapid development of medical imaging technology, the accuracy and efficiency of stroke assessment have been significantly improved. However, traditional image evaluation methods still have some problems. Currently, ischemic stroke is the main type of stroke, accounting for 70-80% of all strokes. Existing automatic segmentation methods, such as models based on convolutional neural networks (CNN) or Transformers, perform poorly in capturing long-distance dependencies between image pixels and have high computational resource requirements, resulting in a heavy computational burden and efficiency bottleneck in practical applications.

[0003] Furthermore, most existing methods for stroke classification and locating the responsible vessel are based on traditional clinical indicators and single-modality imaging data, such as computed tomography (CT) and magnetic resonance imaging (MRI). While these methods can provide some structural information, they cannot comprehensively capture the aging state of cerebrovascular vessels and subtle lesions. In particular, for acute ischemic stroke, accurate localization of early lesions is crucial.

[0004] Therefore, improving the efficiency of stroke assessment to gain golden time for thrombolysis or thrombectomy is an urgent technical problem to be solved. Summary of the Invention

[0005] In view of the above problems, the present invention provides a method, apparatus, device and medium for assisting in stroke assessment that overcomes or at least partially solves the above problems.

[0006] This invention provides a method to assist in stroke assessment, comprising: Receives brain images, clinical vital signs data, pre-hospital emergency records, and historical health data, including CT images and MRI images; A hybrid encoder combining convolutional neural networks and Swin-Transformer is used to extract deep semantic features from the brain images. A gradient boosting tree feature encoder is used to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data. Based on the deep semantic features and the nonlinear correlation features, complementary fusion features are obtained by using residual connections and multi-head attention mechanisms. Based on the fusion features, the stroke type is determined, including ischemic stroke and hemorrhagic stroke; When ischemic stroke is diagnosed, the volume of the core infarct area is calculated.

[0007] Preferably, after receiving brain imaging and clinical vital signs data, wherein the brain imaging includes CT images and MRI images, the method further includes: Spatial centroid alignment, rigid body registration, and grayscale reconstruction are performed on the brain image to obtain the processed brain image.

[0008] Preferably, a hybrid encoder combining a convolutional neural network and a Swin-Transformer is used to extract deep semantic features from the brain images, including: Feature maps of the brain images are extracted using convolutional layers of a convolutional neural network, according to the following formula:

[0009] in, The brain image mentioned above. The convolution kernel is 3×3. The number of convolution kernels, For convolution operations, For bias terms, For feature extraction activation function; The brain images are moved using a displacement window attention mechanism as shown in the following formula to achieve cross-window feature interaction and obtain deep semantic features:

[0010] The query vector for the feature map. The key vector of the feature map. The value vector of the feature map. for transpose, For the mask matrix, For feature dimension, This is the normalized activation function.

[0011] Preferably, a gradient boosting tree feature encoder is used to extract non-linear correlation features between various clinical vital sign data, pre-hospital emergency records, and historical health data, including: A gradient boosting tree feature encoder is used to calculate the residual from the previous iteration. Based on the residuals, train a new regression tree; Based on the new regression tree, the gradient boosting tree feature encoder is updated to obtain the gradient boosting tree iterative algorithm; Based on various clinical vital sign data, pre-hospital emergency records, historical health data, and the gradient boosting tree iterative algorithm, nonlinear correlation features among various clinical vital sign data, pre-hospital emergency records, and historical health data are obtained.

[0012] Preferably, based on the deep semantic features and the nonlinear association features, a residual connection and a multi-head attention mechanism are used to obtain complementary fusion features, including: The deep semantic features and the nonlinear association features are respectively transformed into query vector, key vector and value vector through linear transformation; The query vector, key vector, and value vector are projected into multiple subspaces respectively; The multi-head attention mechanism captures potential associations in different subspaces through each attention head, and then concatenates the captured associations to obtain the first concatenation result. The deep semantic features are concatenated with the nonlinear association features to obtain a second concatenation result; By using residual connection between the first and second stitching results, complementary fusion features are obtained.

[0013] Preferably, when an ischemic stroke is identified, the volume of the core infarct area is calculated, including: Based on the brain images, the pixel set of the core infarct area was determined; The volume of the core infarct region is calculated based on the pixel set of the core infarct region.

[0014] Preferably, when ischemic stroke is determined, after calculating the volume of the core infarct area, the procedure further includes: Based on the aforementioned complementary fusion features, the suitability of using thrombolysis or thrombectomy is evaluated.

[0015] Secondly, the present invention also provides an apparatus for assisting in stroke assessment, comprising: The receiving module is used to receive brain images, clinical vital signs data, pre-hospital emergency records, and historical health data, wherein the brain images include CT images and MRI images; The first extraction module is used to extract deep semantic features of the brain image using a hybrid encoder of convolutional neural network and Swin-Transformer. The second extraction module is used to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data using a gradient boosting tree feature encoder. The fusion module is used to obtain complementary fusion features based on the deep semantic features and the nonlinear correlation features, using residual connections and multi-head attention mechanisms. The classification module is used to determine the stroke type based on the fusion features, wherein the stroke type includes ischemic stroke and hemorrhagic stroke; The measurement module is used to calculate the volume of the core infarct area when the stroke is determined to be ischemic.

[0016] Secondly, the present invention also provides an apparatus for assisting in stroke assessment, comprising: The receiving module is used to receive brain images, clinical vital signs data, pre-hospital emergency records, and historical health data, wherein the brain images include CT images and MRI images; The first extraction module is used to extract deep semantic features of the brain image using a hybrid encoder of convolutional neural network and Swin-Transformer. The second extraction module is used to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data using a gradient boosting tree feature encoder. The fusion module is used to obtain complementary fusion features based on the deep semantic features and the nonlinear correlation features, using residual connections and multi-head attention mechanisms. The classification module is used to determine the stroke type based on the fusion features, wherein the stroke type includes ischemic stroke and hemorrhagic stroke; The measurement module is used to calculate the volume of the core infarct area when the stroke is determined to be ischemic.

[0017] Thirdly, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0018] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0019] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages: This invention provides a method to assist in stroke assessment, comprising: receiving brain images, clinical vital signs data, pre-hospital emergency records, and historical health data, wherein the brain images include CT images and MRI images; employing a lightweight convolutional neural network and a Swin-Transformer hybrid encoder to extract deep semantic features from the brain images; employing a gradient boosting tree feature encoder to extract nonlinear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data; based on the deep semantic features and nonlinear correlation features, employing residual connections and multi-head attention mechanisms to obtain complementary fusion features; based on the fusion features, determining the stroke type, which includes ischemic stroke and hemorrhagic stroke; when ischemic stroke is determined, calculating the volume of the core infarct area; and by effectively fusing multimodal data, overcoming the shortcomings of traditional image analysis that relies on expert experience and is time-consuming, thus shortening the emergency stroke assessment time. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating the steps of a method for assisting stroke assessment in an embodiment of the present invention is shown. Figure 2 A schematic diagram of the structure of a device for assisting stroke assessment in an embodiment of the present invention is shown; Figure 3 A schematic diagram of the structure of a computer device for implementing a method to assist in stroke assessment is shown in an embodiment of the present invention. Detailed Implementation

[0021] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0022] Example 1: Embodiments of the present invention provide a method for assisting stroke classification, such as... Figure 1 As shown, it includes: S101 receives brain images, clinical vital signs data, pre-hospital emergency records, and historical health data. Brain images include CT images and MRI images. S102 uses a hybrid encoder of convolutional neural network and Swin-Transformer to extract deep semantic features from brain images; S103 uses a gradient boosting tree feature encoder to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data. S104, based on deep semantic features and nonlinear association features, adopts residual connection and multi-head attention mechanism to obtain complementary fusion features; S105, Based on fusion features, the stroke type is determined, which includes ischemic stroke and hemorrhagic stroke; S106, when ischemic stroke is diagnosed, the volume of the core infarct area is calculated.

[0023] First, we receive brain imaging, clinical vital signs data, pre-hospital emergency records, and historical health data. Brain imaging includes CT and MRI images.

[0024] In embodiments of the present invention, by receiving multi-modal data and effectively fusing the features of the multi-modal data, the basis for judgment is enriched, thereby improving the accuracy of the auxiliary evaluation.

[0025] The multimodal data includes two categories: imaging data and clinical data. Imaging data refers to brain images, specifically CT and MRI images. Clinical data includes clinical vital signs data, pre-hospital emergency records, and historical health data.

[0026] Clinical vital signs data include blood pressure, blood sugar, and heart rate; pre-hospital emergency records include onset time and symptom description; this historical health data includes electronic medical record information.

[0027] Next, the brain images are preprocessed. Specifically, spatial centroid alignment, rigid body registration, and grayscale reconstruction are performed on the brain images to obtain the processed brain images.

[0028] Because CT images show a cross-section of the brain relative to the bones, while MRI images show the soft tissues of the brain, they target different objects.

[0029] Spatial centroid alignment can be used to calibrate the spatial consistency of multimodal images, ensuring that the anatomical positions of different modalities are matched.

[0030] The specific steps are as follows: First, fix the centroid of one image, then move the centroid of the other image, and then perform a translation transformation to achieve alignment.

[0031] The centroid of a fixed image is represented as follows:

[0032] in, The total number of pixels in a fixed image. , , For the image in the The three-dimensional coordinates of each pixel.

[0033] The centroid of another image is represented as follows:

[0034] in, The total number of pixels in the other image. , , For the first of the other images The three-dimensional coordinates of each pixel.

[0035] Next, the two centroids are aligned through a translation transformation, and the amount of translation is expressed by the following formula:

[0036] in, , , Thus, the centroids of the space are aligned by moving according to this amount.

[0037] Next, rigid body registration is performed to eliminate spatial misalignment caused by differences in image shooting angles and preserve the geometric features of the anatomical structure.

[0038] The affine transformation formula used is as follows:

[0039] in, The original pixel coordinate vector It is a 3×3 rotation matrix that satisfies , . It is a translation vector.

[0040] Example of a rotation matrix (rotation about the z-axis) Angle), then it is represented as:

[0041] The purpose of grayscale reconstruction is to ensure the consistency of image resolution. The specific methods are as follows: Two-dimensional linear interpolation formula (the same principle applies to three-dimensional interpolation):

[0042] in, For non-integer target coordinates, , , , , , This is the coordinate offset. This represents the grayscale value of each pixel in the image.

[0043] The brain image is obtained through the above spatial centroid alignment, rigid body registration and grayscale reconstruction.

[0044] Next, S102 is executed, which uses a hybrid encoder of convolutional neural network and Swin-Transformer to extract deep semantic features from brain images.

[0045] These deep semantic features include lesion texture, blood vessel morphology, and so on.

[0046] This convolutional neural network employs a lightweight structure, including: an input layer, multiple convolutional layers, multiple pooling layers, multiple deconvolutional layers, and an output layer. The multiple convolutional and pooling layers are used to extract features from the input image, and the deconvolutional layers are used to restore the resolution of the feature map.

[0047] Specifically, features are extracted, including: Feature maps of brain images are extracted using convolutional layers of a convolutional neural network, according to the following formula:

[0048] in, Brain imaging, The convolution kernel is 3×3. The number of convolution kernels, For convolution operations, For bias terms, For feature extraction activation function; The brain images are moved using a displacement window attention mechanism as shown in the following formula to achieve cross-window feature interaction and obtain deep semantic features:

[0049] The query vector for the feature map. The key vector of the feature map. The value vector of the feature map. for transpose, For the mask matrix, For feature dimension, This is the normalized activation function.

[0050] Upsampling is performed by deconvolution layers, according to the following formula:

[0051] in, To deconvolution kernel size, The step size doubles the feature map resolution. The transpose convolution operation is an operation that enlarges the spatial size of the feature map by learning weights.

[0052] This yields the deep semantic features of brain images, denoted as... .

[0053] Next, step S103 is executed, employing a gradient boosting tree feature encoder to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data. Specifically, it includes: A gradient boosting tree feature encoder is used to calculate the residual from the previous iteration. Train a new regression tree based on the residuals; Based on the new regression tree, the gradient boosting tree feature encoder is updated to obtain the gradient boosting tree iterative algorithm; Based on various clinical vital sign data, pre-hospital emergency records, historical health data, and gradient boosting tree iterative algorithm, the nonlinear correlation characteristics among various clinical vital sign data, pre-hospital emergency records, and historical health data are obtained.

[0054] The gradient boosting tree (GBDT) feature encoder is used to mine non-linear relationships between clinical data, such as vital signs data and medical history.

[0055] Residual from the previous iteration:

[0056] For the first The output of the model in each iteration, The input data includes clinical vital signs data, pre-hospital emergency records, and historical health data. Corresponding to the actual value, This represents the residual from the previous iteration.

[0057] The new regression tree uses this residual as its learning target and gradually reduces the prediction error through iterative optimization, thus obtaining the new regression tree:

[0058] Next, based on this new regression tree, the gradient boosting tree feature encoder is updated, resulting in the gradient boosting tree feature iterative algorithm:

[0059] in, =0.01 is the learning rate.

[0060] Next, the key features are selected, specifically obtained through the following calculation formula:

[0061] For the first Clinical features (such as blood pressure or onset time, etc.). Clinical features In the Information gain in a tree For the first The degree of contribution of each clinical feature This represents the total number of iterations.

[0062] The key features are used in an iterative gradient boosting tree algorithm to obtain the nonlinear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data, denoted as . .

[0063] Next, S104 is executed, which uses residual connections and multi-head attention mechanisms based on deep semantic features and nonlinear association features to obtain complementary fusion features.

[0064] Specifically, deep semantic features and nonlinear association features are transformed into query vectors, key vectors, and value vectors through linear transformations, respectively. The query vector, key vector, and value vector are projected into multiple subspaces respectively; The multi-head attention mechanism captures potential associations in different subspaces through each attention head, and then concatenates the captured associations to obtain the first concatenation result. The deep semantic features are concatenated with the nonlinear association features to obtain the second concatenation result; By using residual connection between the first and second stitching results, complementary fusion features are obtained.

[0065] In a specific implementation, the query vector is The key vector is The sum vector is .

[0066] The first splicing result is shown in the following calculation formula:

[0067] in, For any attention head to capture the association, The final output projection matrix, , , For the first The projection matrix of each attention head. This is the first splicing result. This is for splicing operations.

[0068] Next, the deep semantic features are concatenated with the nonlinear association features to obtain the second concatenation result, as shown in the following formula:

[0069] in, This is the second splicing result. It is a non-linear correlation feature. It represents deep semantic features.

[0070] Finally, residual connection is used to connect the first and second stitching results to obtain complementary fusion features, which are recorded as follows: .

[0071] Next, S105 is executed to determine the stroke type based on the fusion features. The stroke type includes ischemic stroke and hemorrhagic stroke.

[0072] Specifically, the fused features are input into a classification model to obtain the stroke type, which is obtained through training.

[0073] Because this fusion feature combines effective information from imaging and clinical data, it overcomes the limitations of existing data based on only a single modality, thereby improving the ability to capture the aging state of cerebrovascular system and subtle lesions, especially for acute ischemic stroke, enabling precise localization of early lesions.

[0074] Ischemic stroke is caused by cerebral vascular occlusion leading to cerebral ischemia and hypoxia, accounting for about 85% of all strokes; hemorrhagic stroke is caused by cerebral vascular rupture leading to intracranial or subarachnoid hemorrhage, accounting for about 15%.

[0075] Next, when ischemic stroke is confirmed, the volume of the core infarct area is calculated by executing S106.

[0076] Specifically, based on brain imaging, the pixel set of the core infarct area is determined; Based on this core infarct pixel set, the volume of the core infarct region was calculated.

[0077] In determining the pixel set of the core infarct region, a pre-trained infarct region pixel recognition model is used. This model is trained based on historical brain images. Therefore, the pre-trained infarct region pixel recognition model is used to identify the current brain image, thereby obtaining the pixel set of the core infarct region.

[0078] Next, based on this core infarct pixel set, the volume of the core infarct region is calculated using the following formula:

[0079] in, For the determined core infarct region pixel set, The voxel size is for a single pixel, and this voxel size is determined by the image scanning parameters. The volume of the core infarct region.

[0080] The volume of the core infarct area can be used to predict the subsequent progression of the disease.

[0081] After calculating the volume of the core infarct region, the following steps are also included: Based on complementary fusion characteristics, the suitability of using thrombolysis or thrombectomy is evaluated.

[0082] First, let's introduce the scenarios for thrombolysis and thrombectomy. Thrombolysis primarily uses medication to dissolve existing blood clots; thrombectomy involves surgically removing the blood clot from the blood vessel. Both aim to clear blockages in blood vessels and restore normal blood flow.

[0083] The suitability of thrombolysis or thrombectomy is specifically evaluated using the following calculation formula:

[0084] in, For learnable parameters, As complementary integration features, For bias terms, is the activation function of the neural network.

[0085] exist If the condition is met, it is determined to be compatible; otherwise, it is determined to be incompatible.

[0086] At this point, it's also necessary to combine this data with TEG (thromboelastography). If the TEG data shows a high risk of bleeding (e.g., clot strength < threshold), even... However, thrombolysis compatibility should also be ruled out.

[0087] For embolic occlusions (such as cardiac emboli), the weighting of thrombectomy suitability is increased. The threshold for (thrombectomy probability) is set to 65%, and the obtained... When the value exceeds this threshold, thrombectomy is the preferred procedure.

[0088] If the occlusion is due to intracranial atherosclerosis, the thrombolysis suitability weight is reduced (to avoid platelet aggregation that could worsen the stenosis), the Plyse (probability of thrombolysis) threshold is set to 85%, and it is recommended to first treat with antiplatelet therapy before evaluating thrombectomy.

[0089] Finally, the output results are: thrombolysis first, thrombectomy first, or combined treatment.

[0090] To improve the accuracy of the model classification, a validation set is used to evaluate the loss function during the iteration process. Make a judgment, if:

[0091] At this point, stop training and take... The model parameters of the wheel are used as the final trained model.

[0092] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages: This invention provides a method to assist in stroke assessment, comprising: receiving brain images, clinical vital signs data, pre-hospital emergency records, and historical health data, wherein the brain images include CT images and MRI images; employing a lightweight convolutional neural network and a Swin-Transformer hybrid encoder to extract deep semantic features from the brain images; employing a gradient boosting tree feature encoder to extract nonlinear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data; based on the deep semantic features and nonlinear correlation features, employing residual connections and multi-head attention mechanisms to obtain complementary fusion features; based on the fusion features, determining the stroke type, which includes ischemic stroke and hemorrhagic stroke; when ischemic stroke is determined, calculating the volume of the core infarct area; and by effectively fusing multimodal data, overcoming the shortcomings of traditional image analysis that relies on expert experience and is time-consuming, thus shortening the emergency stroke assessment time.

[0093] Example 2: Based on the same inventive concept, embodiments of the present invention provide a device for assisting in stroke assessment, such as... Figure 2 As shown, it includes: The receiving module 201 is used to receive brain images, clinical vital signs data, pre-hospital emergency records, and historical health data, wherein the brain images include CT images and MRI images; The first extraction module 202 is used to extract deep semantic features of the brain image using a hybrid encoder of convolutional neural network and Swin-Transformer. The second extraction module 203 is used to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records and historical health data using a gradient boosting tree feature encoder. The fusion module 204 is used to obtain complementary fusion features based on the deep semantic features and the nonlinear association features, using residual connections and multi-head attention mechanisms. The classification module 205 is used to determine the stroke type based on the fusion features, wherein the stroke type includes ischemic stroke and hemorrhagic stroke. The calculation module 206 is used to calculate the volume of the core infarct area when the stroke is determined to be ischemic.

[0094] In one alternative implementation, it further includes: a processing module, configured to: Spatial centroid alignment, rigid body registration, and grayscale reconstruction are performed on the brain image to obtain the processed brain image.

[0095] In one optional implementation, the first extraction module 202 is configured to: Feature maps of the brain images are extracted using convolutional layers of a convolutional neural network, according to the following formula:

[0096] in, The brain image mentioned above. The convolution kernel is 3×3. The number of convolution kernels, For convolution operations, For bias terms, For feature extraction activation function; The brain images are moved using a displacement window attention mechanism as shown in the following formula to achieve cross-window feature interaction and obtain deep semantic features:

[0097] The query vector for the feature map. The key vector of the feature map. The value vector of the feature map. for transpose, For the mask matrix, For feature dimension, This is the normalized activation function.

[0098] In one optional implementation, the second extraction module 203 is used for: A gradient boosting tree feature encoder is used to calculate the residual from the previous iteration. Based on the residuals, train a new regression tree; Based on the new regression tree, the gradient boosting tree feature encoder is updated to obtain the gradient boosting tree iterative algorithm; Based on various clinical vital sign data, pre-hospital emergency records, historical health data, and the gradient boosting tree iterative algorithm, nonlinear correlation features among various clinical vital sign data, pre-hospital emergency records, and historical health data are obtained.

[0099] In one alternative implementation, the fusion module 204 is configured to: The deep semantic features and the nonlinear association features are respectively transformed into query vector, key vector and value vector through linear transformation; The query vector, key vector, and value vector are projected into multiple subspaces respectively; The multi-head attention mechanism captures potential associations in different subspaces through each attention head, and then concatenates the captured associations to obtain the first concatenation result. The deep semantic features are concatenated with the nonlinear association features to obtain a second concatenation result; By using residual connection between the first and second stitching results, complementary fusion features are obtained.

[0100] In one alternative implementation, the measurement module 206 is used for: Based on the brain images, the pixel set of the core infarct area was determined; The volume of the core infarct region is calculated based on the pixel set of the core infarct region.

[0101] In an alternative implementation, it further includes: an evaluation module, used for: Based on the aforementioned complementary fusion features, the suitability of using thrombolysis or thrombectomy is evaluated.

[0102] Example 3: Based on the same inventive concept, embodiments of the present invention provide a computer device, such as... Figure 3 As shown, it includes a memory 304, a processor 302, and a computer program stored in the memory 304 and executable on the processor 302. When the processor 302 executes the program, it implements the steps of the above-described method for assisting in stroke assessment.

[0103] Among them, Figure 3 In this document, a bus architecture (represented by bus 300) is used. Bus 300 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 302 and memory represented by memory 304. Bus 300 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 306 provides an interface between bus 300 and receiver 301 and transmitter 303. Receiver 301 and transmitter 303 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium.

[0104] The processor 302 is responsible for managing the bus 300 and general processing, while the memory 304 can be used to store the data used by the processor 302 when performing operations.

[0105] Example 4: Based on the same inventive concept, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for assisting in stroke assessment.

[0106] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0107] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0108] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are explicitly recited in each embodiment. Rather, as reflected in each embodiment, inventive aspects lie in fewer than all features of the single foregoing disclosed embodiment. Therefore, the claims, following the detailed description, are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0109] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0110] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the specific implementation, any of the claimed embodiments can be used in any combination.

[0111] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the apparatus or computer device assisting in stroke assessment according to embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0112] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A method for assisting in stroke assessment, characterized in that, include: Receives brain images, clinical vital signs data, pre-hospital emergency records, and historical health data, including CT images and MRI images; A hybrid encoder combining convolutional neural networks and Swin-Transformer is used to extract deep semantic features from the brain images. A gradient boosting tree feature encoder is used to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data. Based on the deep semantic features and the nonlinear correlation features, complementary fusion features are obtained by using residual connections and multi-head attention mechanisms. Based on the fusion features, the stroke type is determined, including ischemic stroke and hemorrhagic stroke; When ischemic stroke is diagnosed, the volume of the core infarct area is calculated.

2. The method as described in claim 1, characterized in that, After receiving brain imaging and clinical vital signs data, including CT and MRI images, the following are also included: Spatial centroid alignment, rigid body registration, and grayscale reconstruction are performed on the brain image to obtain the processed brain image.

3. The method as described in claim 1 or 2, characterized in that, A hybrid encoder combining convolutional neural networks and Swin-Transformer is used to extract deep semantic features from the brain images, including: Feature maps of the brain images are extracted using convolutional layers of a convolutional neural network, according to the following formula: ； in, The brain image mentioned above. The convolution kernel is 3×3. The number of convolution kernels, For convolution operations, For bias terms, The activation function for feature extraction; The brain images are moved using a displacement window attention mechanism as shown in the following formula to achieve cross-window feature interaction and obtain deep semantic features: ； The query vector for the feature map. The key vector of the feature map. The value vector of the feature map. for transpose, For the mask matrix, For feature dimension, This is the normalized activation function.

4. The method as described in claim 1, characterized in that, A gradient boosting tree feature encoder is used to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data, including: A gradient boosting tree feature encoder is used to calculate the residual from the previous iteration. Based on the residuals, train a new regression tree; Based on the new regression tree, the gradient boosting tree feature encoder is updated to obtain the gradient boosting tree iterative algorithm; Based on various clinical vital sign data, pre-hospital emergency records, historical health data, and the gradient boosting tree iterative algorithm, nonlinear correlation features among various clinical vital sign data, pre-hospital emergency records, and historical health data are obtained.

5. The method as described in claim 1, characterized in that, Based on the deep semantic features and the nonlinear correlation features, complementary fusion features are obtained by employing residual connections and multi-head attention mechanisms, including: The deep semantic features and the nonlinear association features are respectively transformed into query vector, key vector and value vector through linear transformation; The query vector, key vector, and value vector are projected into multiple subspaces respectively; The multi-head attention mechanism captures potential associations in different subspaces through each attention head, and then concatenates the captured associations to obtain the first concatenation result. The deep semantic features are concatenated with the nonlinear association features to obtain a second concatenation result; By using residual connection between the first and second stitching results, complementary fusion features are obtained.

6. The method as described in claim 1, characterized in that, When ischemic stroke is diagnosed, the volume of the core infarct area is calculated, including: Based on the brain images, the pixel set of the core infarct area was determined; The volume of the core infarct region is calculated based on the pixel set of the core infarct region.

7. The method as described in claim 1, characterized in that, When ischemic stroke is diagnosed, after calculating the volume of the core infarct area, the following steps are also taken: Based on the aforementioned complementary fusion features, the suitability of using thrombolysis or thrombectomy is evaluated.

8. A device for assisting in stroke assessment, characterized in that, include: The receiving module is used to receive brain images, clinical vital signs data, pre-hospital emergency records, and historical health data, wherein the brain images include CT images and MRI images; The first extraction module is used to extract deep semantic features of the brain image using a hybrid encoder of convolutional neural network and Swin-Transformer. The second extraction module is used to extract non-linear correlation features between various clinical vital signs data, pre-hospital emergency records, and historical health data using a gradient boosting tree feature encoder. The fusion module is used to obtain complementary fusion features based on the deep semantic features and the nonlinear correlation features, using residual connections and multi-head attention mechanisms. The classification module is used to determine the stroke type based on the fusion features, wherein the stroke type includes ischemic stroke and hemorrhagic stroke; The measurement module is used to calculate the volume of the core infarct area when the stroke is determined to be ischemic.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images