Pulmonary hypertension detection method fusing image features and tricuspid regurgitation velocity

Abstract

The application provides a pulmonary hypertension detection method fusing image features and tricuspid regurgitation velocity, which is used for creating a double-model fusion architecture when combining computer tomography images, B-ultrasonic tricuspid regurgitation velocity and neural network technology to detect pulmonary hypertension, and continuously giving a series of optimization configuration schemes from details, so that efficient and high-precision pulmonary hypertension detection effect can be realized, the workload and work error required by relevant medical staff can be effectively reduced in actual application, artificial cost is significantly reduced, so that high-quality data support can be provided, and the diagnosis and treatment of pulmonary hypertension can be better promoted, and the application prospect is better.

Description

Technical Field

[0001] This application relates to the field of medical technology, specifically to a method for detecting pulmonary hypertension by fusing imaging features with tricuspid regurgitation velocity. Background Technology

[0002] Computed tomography (CT) images have high density resolution, meaning that different tissues exhibit good contrast in CT images. This makes the pulmonary artery and its surrounding structures clearer, facilitating the identification of pulmonary artery dilation, changes in lung parenchyma, calcifications, and changes in cardiac morphology—all signs associated with pulmonary hypertension. CT provides particularly intuitive and quantitative data support when assessing signs such as the diameter of the main pulmonary artery, right ventricular hypertrophy, and pericardial effusion. While conventional CT scans cannot currently visualize changes in intravascular pressure in the pulmonary vessels, indirect indicators such as the ratio of pulmonary artery diameter to descending aorta diameter (PA / Ao ratio) can still provide a preliminary assessment of the likelihood of pulmonary hypertension.

[0003] The multi-angle, multi-section imaging capabilities of computed tomography (CT) overcome the limitations of overlapping tissue structures in traditional X-ray images, thus enabling more accurate observation of details such as pulmonary vascular pathways, bronchial structures, and changes in lung texture. This makes computed tomography pulmonary artery imaging an important tool for assessing pulmonary hypertension. By combining high-resolution computed tomography (HRCT) with computed tomography angiography (CTA), it is possible not only to observe the course and branch clarity of the pulmonary arteries but also to assess whether parenchymal lung lesions are the underlying cause of pulmonary hypertension, such as pulmonary interstitial fibrosis and chronic thromboembolism.

[0004] Magnetic resonance imaging (MRI) has advantages in soft tissue contrast and is worse than computed tomography (CT) for assessing early changes in cardiac structure, especially in assessing right ventricular function. However, it is not as clear as CT in displaying lung structures, and the image acquisition time is longer. It is also difficult to widely implement due to patient compliance and scanning conditions.

[0005] However, the spatial resolution of computed tomography (CT) images is still insufficient compared to magnetic resonance imaging (MRI) and X-rays. That is, its ability to express details of a single pixel is limited, making it difficult to accurately quantify changes in the diameter of small blood vessels and microcalcifications, thus limiting the detection of early pulmonary hypertension lesions and minor anatomical abnormalities.

[0006] In addition, the computer tomography of pulmonary arterial hypertension shows many indirect signs, and there is no unified gold standard image feature, which makes the image-based judgment easily affected by human experience.

[0007] Therefore, although in the background of widely introducing Artificial Intelligence (AI) technology in medical technology research and clinical work to realize the prediction of corresponding diseases, how to realize high-performance pulmonary arterial hypertension detection based on computer tomography images is still a major technical problem. SUMMARY

[0008] The present application provides a pulmonary arterial hypertension detection method fusing image features and tricuspid regurgitation velocity, which is used to create a double-model fusion architecture when combining computer tomography images, B-ultrasound tricuspid regurgitation velocity and neural network technology to detect pulmonary arterial hypertension, and a series of optimization configuration schemes are further given in detail, thereby realizing efficient and high-precision pulmonary arterial hypertension detection effect, which can effectively reduce the workload and work error of related medical staff in actual application, significantly reduce the labor cost, thereby providing high-quality data support, helping to better promote the diagnosis and treatment of pulmonary arterial hypertension, and having a better application prospect.

[0009] In a first aspect, the present application provides a multi-modal pulmonary arterial hypertension detection method fusing medical image features and tricuspid regurgitation velocity, the method comprising:

[0010] obtaining a to-be-detected lung computer tomography image and a to-be-detected B-ultrasound tricuspid regurgitation velocity corresponding to a current pulmonary arterial hypertension detection task;

[0011] inputting the to-be-detected lung computer tomography image and the to-be-detected B-ultrasound tricuspid regurgitation velocity into a pre-configured pulmonary arterial hypertension detection model, wherein the pulmonary arterial hypertension detection model comprises a tissue site detection model part and a pulmonary arterial hypertension detection model part, the tissue site detection model part is used to perform classification and identification of target tissue sites related to pulmonary arterial hypertension on the input lung computer tomography image, and the pulmonary arterial hypertension detection model part is used to perform pulmonary arterial hypertension detection processing on the classification and identification result output by the tissue site detection model part in combination with the input B-ultrasound tricuspid regurgitation velocity, and the obtained pulmonary arterial hypertension detection processing result is taken as the model output;

[0012] extracting a target pulmonary arterial hypertension detection result output by the pulmonary arterial hypertension detection model.

[0013] In a second aspect, the present application provides a pulmonary arterial hypertension detection device fusing image features and tricuspid regurgitation velocity, the device comprising:

[0014] The acquisition unit is configured to acquire a to-be-detected lung computed tomography image and a to-be-detected B-ultrasonic tricuspid regurgitation velocity corresponding to a current pulmonary hypertension detection task.

[0015] The detection unit is configured to input the to-be-detected lung computed tomography image and the to-be-detected B-ultrasonic tricuspid regurgitation velocity into a pre-configured pulmonary hypertension detection model, wherein the pulmonary hypertension detection model comprises a tissue site detection model part and a pulmonary hypertension detection model part, the tissue site detection model part is configured to perform classification recognition of a target tissue site related to a pulmonary hypertension condition on the lung computed tomography image input into the model, and the pulmonary hypertension detection model part is configured to perform pulmonary hypertension detection processing on the classification recognition result output by the tissue site detection model part in combination with the B-ultrasonic tricuspid regurgitation velocity input into the model, and a pulmonary hypertension detection processing result obtained by the processing is taken as a model output.

[0016] The extraction unit is configured to extract a target pulmonary hypertension detection result output by the pulmonary hypertension detection model.

[0017] In a third aspect, the present application provides a processing device, comprising a processor and a memory, the memory storing a computer program, and the processor executes the method provided in the first aspect of the present application when invoking the computer program in the memory.

[0018] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a plurality of instructions, the instructions being adapted to be loaded by a processor to execute the method provided in the first aspect of the present application.

[0019] From the above, the present application has the following beneficial effects:

[0020] For the pulmonary hypertension detection target of fusing image features and tricuspid regurgitation velocity, when the present application combines a computed tomography image, a B-ultrasonic tricuspid regurgitation velocity and a neural network technology to perform pulmonary hypertension detection, a double-model fusion architecture is created, and a series of optimization configuration schemes are further given in details, thereby realizing efficient and high-precision pulmonary hypertension detection effect, effectively reducing the workload and work error of relevant medical staff in actual application, significantly reducing the labor cost, thereby providing high-quality data support, helping to better promote the diagnosis and treatment of pulmonary hypertension, and having a better application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 A flowchart of a method for detecting pulmonary hypertension by fusing image features and tricuspid regurgitation velocity according to the present application;

[0023] Figure 2 A structural diagram of a device for detecting pulmonary hypertension by fusing image features and tricuspid regurgitation velocity according to the present application;

[0024] Figure 3 A structural diagram of a processing device according to the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0026] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or modules does not necessarily have to be limited to those steps or modules clearly listed, but can include other steps or modules that are not clearly listed or inherent to the process, method, product, or device. The naming or numbering of the steps appearing in the present application does not mean that the steps in the method flow must be executed in the time / chronological order indicated by the naming or numbering, and the named or numbered flow steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0027] The division of the modules appearing in the present application is a logical division. In actual application, another division manner can be used, for example, a plurality of modules can be combined or integrated in another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the modules shown or discussed can be through an interface, and the indirect coupling or communication connection between the modules can be electrical or other similar forms, which are not limited in the present application. In addition, the modules or sub-modules described as separate components can or can not be physically separated, can or can not be physical modules, or can be distributed in a plurality of circuit modules. According to actual needs, part or all of the modules can be selected to achieve the purpose of the present application scheme.

[0028] Before introducing the pulmonary hypertension detection method provided by the present application which fuses image features and tricuspid regurgitation velocity, first introduce the background content involved in the present application.

[0029] The pulmonary hypertension detection method, device and computer readable storage medium provided by the present application which fuses image features and tricuspid regurgitation velocity can be applied to a processing device, which is used to create a double-model fusion architecture when combining computer tomography images and neural network technology to detect pulmonary hypertension, and a series of optimization configuration schemes are further given in detail. Therefore, efficient and high-precision pulmonary hypertension detection effect can be achieved, which can effectively reduce the workload and work error of related medical staff in actual application, significantly reduce the labor cost, and thus provide high-quality data support, which is helpful to better promote the diagnosis and treatment of pulmonary hypertension, and has a better application prospect.

[0030] The pulmonary hypertension detection method which fuses image features and tricuspid regurgitation velocity mentioned in the present application can be executed by a pulmonary hypertension detection device which fuses image features and tricuspid regurgitation velocity, or a server, a physical host or a user equipment (UE) of different types of processing devices integrated with the pulmonary hypertension detection device. The pulmonary hypertension detection device which fuses image features and tricuspid regurgitation velocity can be realized by hardware or software. The UE can be a terminal device such as a smart phone, a tablet computer, a notebook computer, a desktop computer or a personal digital assistant (PDA). The processing device can be set by a device cluster.

[0031] It is understandable that the solution in this application is mainly based on the data processing carried out on the basis of the already acquired or ready-made computed tomography images. Therefore, the processing equipment that implements the pulmonary hypertension detection method of fusing image features and tricuspid regurgitation velocity of this application, or the processing equipment that carries the corresponding application service of the pulmonary hypertension detection method of fusing image features and tricuspid regurgitation velocity of this application, usually only needs to meet the required data processing capabilities. The specific equipment type and equipment deployment form can be flexibly configured according to the actual situation.

[0032] In specific applications, if the acquisition and processing of specific computed tomography (CT) images are involved, i.e., CT scan processing, then the corresponding CT scan equipment can be included in the scope of the processing equipment of this application, or the CT scan service can be provided as an external linkage device.

[0033] Furthermore, if further processing is involved, such as displaying results or processing progress, it is obviously necessary to make further adaptive configurations for the equipment type and deployment form of the processing equipment based on the actual situation.

[0034] Taking the requirement to configure display functions as an example, the processing device can display specific content through its own configured display screen (including touch screen), external display devices, or other devices with display screens.

[0035] The following section introduces the pulmonary hypertension detection method based on fused imaging features and tricuspid regurgitation velocity provided in this application.

[0036] First, refer to Figure 1 , Figure 1 This paper illustrates a flowchart of a pulmonary hypertension detection method that integrates imaging features and tricuspid regurgitation velocity, as described in this application. The pulmonary hypertension detection method that integrates imaging features and tricuspid regurgitation velocity provided in this application may specifically include the following steps S101 to S103:

[0037] Step S101: Obtain the computed tomography image of the lung to be detected and the tricuspid regurgitation velocity of the ultrasound corresponding to the current pulmonary hypertension detection task;

[0038] It is easy to understand that, corresponding to the case that the present application designs a novel pulmonary hypertension detection model based on artificial intelligence technology, in the specific application of the pulmonary hypertension detection model, the current to-be-processed lung computed tomography image needs to be obtained, for the convenience of description, the current to-be-processed image is recorded as a to-be-detected lung computed tomography image (which can be simply referred to as a to-be-detected CT image) and a to-be-detected B-ultrasound tricuspid regurgitation velocity (Tricuspid Regurgitation velocity, TR velocity), which is an existing clinical indicator for evaluating cardiopulmonary function.

[0039] In addition, it can also be seen that, in actual application, the application of the present application based on the pulmonary hypertension detection model is usually initiated in the form of a task.

[0040] As for the acquisition and processing of the to-be-detected lung computed tomography image and the to-be-detected B-ultrasound tricuspid regurgitation velocity, it can be directly extracted from the task information of the current pulmonary hypertension detection task, or it can be extracted according to the indication and guidance of the task information of the current pulmonary hypertension detection task, or it can be obtained by real-time computed tomography processing and B-ultrasound tricuspid regurgitation velocity examination processing (i.e. real-time acquisition processing) according to the indication and guidance of the task information of the current pulmonary hypertension detection task. The specific acquisition approach is flexible and can be configured according to actual conditions.

[0041] Step S102, input the to-be-detected lung computed tomography image and the to-be-detected B-ultrasound tricuspid regurgitation velocity into the pre-selected pulmonary hypertension detection model, wherein the pulmonary hypertension detection model includes a tissue site detection model part and a pulmonary hypertension detection model part, the tissue site detection model part is used to classify and identify the target tissue site related to the pulmonary hypertension disease of the input lung computed tomography image of the model, and the pulmonary hypertension detection model part is used to combine the classification and identification result output by the tissue site detection model part with the input B-ultrasound tricuspid regurgitation velocity of the model to carry out pulmonary hypertension detection processing, and the obtained pulmonary hypertension detection processing result is taken as the model output;

[0042] It should be understood that, for the pulmonary hypertension detection model specially designed by the present application, it is specifically divided into two parts:

[0043] The first part is the tissue site detection model part, the input of the tissue site detection model part is the input lung computed tomography image of the model, which is used to classify and identify the target tissue site related to the pulmonary hypertension disease of the input lung computed tomography image of the model;

[0044] The second part is a pulmonary hypertension detection model part, and an output of the pulmonary hypertension detection model part is an output of the pulmonary hypertension detection model and an input B-ultrasound tricuspid regurgitation speed of the combination model (in some special cases, other types of clinical parameters can also be involved), and the pulmonary hypertension detection model part performs pulmonary hypertension detection processing on the classification recognition result output by the tissue site detection model part and the input B-ultrasound tricuspid regurgitation speed of the combination model.

[0045] Among them, the two are usually made by adopting different types of model architectures, so as to correspond to different prediction targets, so as to achieve more adaptive and more refined processing effects.

[0046] For the former, that is, the tissue site detection model part, it can be understood that in actual situations, the computed tomography image multi-volume data, the computed tomography image of a patient often contains hundreds of slices, and the slices that really contain effective diagnostic information are very limited. Therefore, in the traditional processing mode, the screening and labeling process needs to rely on professional doctors to spend a lot of time to complete, the labeling cost is high and the consistency is difficult to guarantee, and combined with the limitation of the spatial resolution of the computed tomography image, it is easy to cause the lesion boundary to be blurred and the recognition accuracy to be reduced, which further affects the performance of the AI model. In this case, the working goal of the tissue site detection model part is to efficiently and accurately screen and identify the target tissue site highly related to the pulmonary hypertension detection logic designed in the present application from a large number of slices contained in the model input lung computed tomography image.

[0047] In a popular way, the computed tomography image has more noise, which is usually composed of a region of interest (Region of Interest, ROI) and a background region, and the ROI region often contains key diagnostic information. Therefore, the tissue site detection model part (that is, the detection algorithm) detects a very small amount of ROI region from a large number of front slices, and then the pulmonary hypertension detection model part analyzes and processes the lesion.

[0048] And for the latter, that is, the pulmonary hypertension detection model part, it is based on the highly simplified ROI region given by the tissue site detection model part, combined with the input B-ultrasound tricuspid regurgitation speed of the combination model, so that the computing resources can be efficiently utilized in a short time to finely capture the lesion image features, and then the corresponding pulmonary hypertension situation can be accurately inferred and predicted.

[0049] In this way, the dual-model fusion architecture or dual-layer AI inference mechanism described above is used to achieve the high-performance pulmonary hypertension detection goal.

[0050] In step S103, the target pulmonary hypertension detection result output by the pulmonary hypertension detection model is extracted.

[0051] It can be understood that after the corresponding pulmonary hypertension detection processing is completed for the current to-be-detected lung computed tomography image and the to-be-detected B-ultrasonic tricuspid regurgitation velocity by the pulmonary hypertension detection model, obviously, the target pulmonary hypertension detection result corresponding to the two of the front to-be-detected lung computed tomography image and the to-be-detected B-ultrasonic tricuspid regurgitation velocity output by the pulmonary hypertension detection model (specifically, the pulmonary hypertension detection model part configured in the second layer in the pulmonary hypertension detection model) can be extracted.

[0052] At this time, further processing can be performed according to the corresponding data application requirements.

[0053] For example, for the target pulmonary hypertension detection result, local storage, off-site storage, result display, output of a detection completion prompt, result pushing, or further data analysis processing, etc. can be performed. The specific processing content can be flexibly adjusted according to the preconfigured and real-time configured data application strategy, so as to meet the diversified data application requirements in actual situations and better assist the diagnosis and treatment of pulmonary hypertension from various dimensions.

[0054] As can be seen from the above Figure 1 embodiments, for the pulmonary hypertension detection target of fusing image features and tricuspid regurgitation velocity, the present application combines computed tomography images, B-ultrasonic tricuspid regurgitation velocity, and neural network technology to perform pulmonary hypertension detection, creates a double-model fusion architecture, and further gives a series of optimization configuration schemes from details, thereby achieving efficient and high-precision pulmonary hypertension detection effect. In actual application, the workload and work error required by related medical staff can be effectively reduced, the artificial cost is significantly reduced, high-quality data support can be provided, and the diagnosis and treatment of pulmonary hypertension can be better promoted, which has a better application prospect.

[0055] The steps of the above Figure 1 embodiments and their possible implementation in actual application will be described in detail.

[0056] As an exemplary embodiment, for the tissue site detection model part configured in the first layer in the pulmonary hypertension detection model, the target tissue site it specifically aims at, that is, the ROI region, can specifically include the pulmonary artery, the descending aorta, the left ventricle, and the right ventricle, etc.

[0057] As for these specific tissue sites, the pulmonary hypertension detection model part configured in the second layer of the pulmonary hypertension detection model in the model can specifically analyze the pulmonary hypertension related lesion image features such as pulmonary artery dilation (corresponding to the diameter of the main trunk of the pulmonary artery), pulmonary parenchyma change (including calcified lesions, pulmonary interstitial fibrosis, chronic thromboembolism), lung texture change, and heart shape change (including right ventricular hypertrophy and pericardial effusion) for determining the specific pulmonary hypertension condition in the next step.

[0058] Further, for the specific neural network architecture involved in the two-layer model part of the pulmonary hypertension detection model, the present application also gives a specific implementation scheme from the detail level.

[0059] Specifically, as an exemplary embodiment, on the one hand, the tissue site detection model part configured in the first layer of the pulmonary hypertension detection model specifically adopts the HydraNet model, and on the other hand, the pulmonary hypertension detection model part configured in the second layer of the pulmonary hypertension detection model specifically adopts the multi-modal convolutional neural network-vision transformer (CNN-ViT) model fusing image features and tricuspid regurgitation velocity.

[0060] More specifically, as mentioned earlier, most of the computer tomography image content is difficult to help in the judgment of pulmonary hypertension, and there are also problems of low contrast and unclear edges, which brings great difficulty to lesion segmentation and disease diagnosis, so in the design of the present application scheme, it is necessary to start data enhancement and screening processing of the computer tomography image to improve the accuracy.

[0061] The present application specifically adopts the HydraNet model to classify and identify the computer tomography image related to pulmonary hypertension, and accurately locates each tissue site (pulmonary artery, descending aorta, left and right ventricles...) from hundreds of computer tomography slices to determine the corresponding ROI area and background area.

[0062] As for the HydraNet model, it is a multi-branch and multi-scale convolutional neural network architecture that can extract information at different levels and scales in the image at the same time, and is suitable for medical image segmentation tasks with complex structure or fuzzy boundary. It introduces multiple decoding branches on the basis of the main network, each branch decodes features at different levels, thereby realizing more fine spatial information recovery. Its advantage lies in enhancing the expression ability of different resolution features in the image and improving the recognition ability of the model for small target structures (such as pulmonary artery), and it is particularly suitable for accurate depiction of blood vessel boundaries and morphology in computer tomography images.

[0063] For the subsequent lesion analysis process for the ROI region, the application specifically uses a multi-modal CNN-ViT model. Compared with traditional convolutional neural network (CNN) detection methods such as Faster R-CNN and YOLOv5, the image detection algorithm has significantly improved in complex structure recognition and global feature modeling.

[0064] The main advantage is that it does not rely on local receptive field convolution operations, but divides the image into fixed-size patches (i.e., a continuous time segment involving two key parameters: patch length and stride), which are then input into the Transformer module as input sequences after linear embedding. This structure makes the model more efficient in handling long-distance pixel relationships, especially suitable for detecting low-contrast, fuzzy boundary, or severely overlapping structure areas in computed tomography images, such as complex anatomical areas such as pulmonary arteries and ventricular walls.

[0065] In this way, the above two specific model architectures further enhance the performance of the dual-model fusion architecture or dual-layer AI inference mechanism designed by the application for pulmonary hypertension detection.

[0066] Among them, the multi-modal CNN-ViT model includes a CNN branch and a ViT branch. As an example, the CNN branch can specifically use a ResNet-50 network, which is responsible for extracting local texture and spatial features. More specifically, the ResNet-50 network is responsible for extracting deep visual features of the classification and recognition results of the target tissue sites related to pulmonary hypertension from the output of the tissue site detection model (HydraNet model), and continues to model global dependencies through the ViT branch. Finally, the image representations output by the two branches and the standardized model input B-mode echocardiography tricuspid regurgitation velocity are input into a multilayer perceptron (MLP) for final feature fusion and classification.

[0067] In addition, it can be understood that the pulmonary hypertension detection model can also involve corresponding training before being put into actual use. For this purpose, as an exemplary embodiment, the method of the application can also include a corresponding model training step, namely:

[0068] Obtain sample lung computed tomography images and sample B-mode echocardiography tricuspid regurgitation velocity;

[0069] Corresponding labels are configured for the sample lung computed tomography images and the sample B-ultrasonic tricuspid regurgitation velocity of the sample;

[0070] The pulmonary hypertension detection model is trained through the sample lung computed tomography images and the sample B-ultrasonic tricuspid regurgitation velocity.

[0071] The sample lung computed tomography images can be images collected from real patients (involving real-time image collection processing or ready image extraction processing), or images obtained by further modification based on real images, or even directly drawn or generated images, so as to correspond to diversified sample configuration requirements and cover various pulmonary hypertension conditions as much as possible, especially rare pulmonary hypertension conditions in actual situations. On the other hand, the sample B-ultrasonic tricuspid regurgitation velocity is the same, and corresponds to the same real or virtual patient as the sample lung computed tomography images.

[0072] For the subsequent labeling process, it is usually completed by manual or manual, or an automatic labeling tool can be used to complete it. The automatic labeling tool needs to be pre-configured with the corresponding automatic labeling logic before use.

[0073] In this way, based on the configured training samples, i.e., the sample lung computed tomography images and the sample B-ultrasonic tricuspid regurgitation velocity that have been labeled, the specific model training can be carried out under the corresponding model training scheme, and the trained model that can be put into actual use is referred to as the pulmonary hypertension detection model involved in the present application.

[0074] The model training scheme and loss function used in the model training process can be understood as being able to use existing schemes, further optimize existing schemes, or be novel schemes researched by oneself, and can be configured as needed.

[0075] As an example, the sample lung computed tomography images and the sample B-ultrasonic tricuspid regurgitation velocity can be randomly divided into a training set and a validation set in a ratio of 3:1. The training set is used to train the model, and the validation set is used to evaluate the accuracy of the classification performance of the constructed model.

[0076] In addition, for the pulmonary hypertension detection model part configured in the second layer in the pulmonary hypertension detection model, the present application also provides a more practical implementation scheme.

[0077] Specifically, as an exemplary embodiment, the training process of the pulmonary hypertension detection model part can include the following processing content:

[0078] For the pre-trained multi-modal CNN-ViT model, load the pre-trained weights and fine-tune them through the transfer learning method.

[0079] It can be understood that, in combination with the pre-training mechanism and the fine-tuning mechanism based on the transfer learning method, it is helpful to achieve more convenient and efficient model training effect for the pulmonary hypertension detection model part.

[0080] As an example, in the deep learning-based image detection model, the PyTorch platform is used for model configuration in the present application, and the multi-modal CNN-ViT model pre-trained by the ImageNet dataset is used to configure the pulmonary hypertension detection model part of the present application. In the retraining process, the model initializes the pre-trained weights, and fine-tunes the multi-modal CNN-ViT model through the transfer learning method to adapt to the ROI region recognition task involved in the present application. When the loss function on the validation set no longer decreases, the training process is stopped under the condition that the network architecture remains unchanged.

[0081] At the same time, in terms of details, the working process of the pulmonary hypertension detection model can also include, as an exemplary embodiment:

[0082] Before inputting the classification and recognition result output by the tissue site detection model part into the pulmonary hypertension detection model part, the ROI region image corresponding to the classification and recognition result is subjected to contrast enhancement processing.

[0083] It can be understood that the present application inserts a contrast enhancement processing link between the tissue site detection model part and the pulmonary hypertension detection model part, to strengthen the corresponding image edge features at the detail level. Then, the pulmonary hypertension detection model part further expands the specific processing of whether there is a lesion according to the image size and its characteristics, more efficiently and accurately.

[0084] In addition, the present application also gives a set of practical implementation scheme for the subsequent result display link that can be involved, and correspondingly, as an exemplary embodiment, the method of the present application can also include:

[0085] Display the target pulmonary hypertension detection result, wherein the target pulmonary hypertension detection result further includes the B-ultrasound tricuspid regurgitation velocity to be detected and the corresponding lesion image features on the basis of the pulmonary hypertension category recognition result, wherein the lesion image features specifically involve the image features of pulmonary artery dilation, pulmonary parenchymal change, heart shape change or chronic thromboembolism.

[0086] It can be seen that, under the condition of whether the output exists pulmonary hypertension, if there is pulmonary hypertension, the pulmonary hypertension detection result given by the pulmonary hypertension detection model can give the specific pulmonary hypertension category recognition result (such as pulsatile pulmonary hypertension, left heart disease caused pulmonary hypertension, pulmonary disease caused pulmonary hypertension, and chronic thromboembolic pulmonary hypertension), and also give the initial B-ultrasound tricuspid regurgitation velocity to be detected and the corresponding lesion characteristics, so as to provide better reference effect by providing more rich image display content, so that medical staff can more intuitively and accurately know the specific situation, and then help to more accurately promote the next step of diagnosis and treatment work in the first time.

[0087] Finally, for the above scheme content, in general, the present application focuses on the fusion of image features and tricuspid regurgitation velocity in the detection of pulmonary hypertension, and solves a series of problems such as data acquisition, standardized annotation, sample expansion and model robustness improvement, so as to obtain a high-performance pulmonary hypertension detection model, which helps to promote the research of AI recognition of pulmonary hypertension based on computer tomography images to practical application.

[0088] The above is an introduction to the pulmonary hypertension detection method provided by the present application, which fuses image features and tricuspid regurgitation velocity. In order to better implement the pulmonary hypertension detection method provided by the present application, which fuses image features and tricuspid regurgitation velocity, the present application also provides a pulmonary hypertension detection device which fuses image features and tricuspid regurgitation velocity from the functional module perspective.

[0089] Referring to Figure 2 , Figure 2 is a structural schematic diagram of the pulmonary hypertension detection device provided by the present application, which fuses image features and tricuspid regurgitation velocity. In the present application, the pulmonary hypertension detection device 200 which fuses image features and tricuspid regurgitation velocity can specifically include the following structures:

[0090] The acquisition unit 201 is configured to acquire a to-be-detected pulmonary CT image and a to-be-detected B-ultrasound tricuspid regurgitation velocity corresponding to a current pulmonary hypertension detection task;

[0091] The detection unit 202 is configured to input the to-be-detected lung computed tomography image and the to-be-detected B-ultrasonic tricuspid regurgitation velocity into a pre-configured pulmonary arterial hypertension detection model, wherein the pulmonary arterial hypertension detection model comprises a tissue site detection model part and a pulmonary arterial hypertension detection model part, the tissue site detection model part is configured to perform classification and identification of a target tissue site related to a pulmonary arterial hypertension condition on the lung computed tomography image input into the model, and the pulmonary arterial hypertension detection model part is configured to perform pulmonary arterial hypertension detection processing on the classification and identification result output by the tissue site detection model part in combination with the B-ultrasonic tricuspid regurgitation velocity input into the model, and the obtained pulmonary arterial hypertension detection processing result is taken as the model output.

[0092] The extraction unit 203 is configured to extract the target pulmonary arterial hypertension detection result output by the pulmonary arterial hypertension detection model.

[0093] In another exemplary embodiment, the target tissue site specifically includes a pulmonary artery, a descending aorta, a left ventricle, and a right ventricle.

[0094] In another exemplary embodiment, the tissue site detection model part specifically adopts a HydraNet model, and the pulmonary arterial hypertension detection model part specifically adopts a multi-modal CNN-ViT model.

[0095] In another exemplary embodiment, the multi-modal CNN-ViT model comprises a CNN branch and a ViT branch, the CNN branch specifically adopts a ResNet-50 network, is responsible for extracting deep visual features of the classification and identification result output by the tissue site detection model, and continues to model global dependency relationship through the ViT branch, and finally inputs the image representations output by the two branches and the B-ultrasonic tricuspid regurgitation velocity input into the model after standardization processing into a multi-layer perceptron for final feature fusion and classification.

[0096] In another exemplary embodiment, the device further comprises a training unit 204 configured to:

[0097] Obtain a sample lung computed tomography image and a sample B-ultrasonic tricuspid regurgitation velocity;

[0098] Configure corresponding labels for the sample lung computed tomography image and the sample B-ultrasonic tricuspid regurgitation velocity;

[0099] Train the pulmonary arterial hypertension detection model through the labeled sample lung computed tomography image and the sample B-ultrasonic tricuspid regurgitation velocity.

[0100] In another exemplary embodiment, the working process of the pulmonary arterial hypertension detection model further comprises:

[0101] Before inputting the classification recognition result output by the tissue site detection model part into the pulmonary hypertension detection model part, the ROI region image corresponding to the classification recognition result is subjected to contrast enhancement processing.

[0102] In yet another exemplary embodiment, the apparatus further comprises a presentation unit 205 configured to:

[0103] present the target pulmonary hypertension detection result, wherein the target pulmonary hypertension detection result further comprises the B-ultrasound tricuspid regurgitation velocity to be detected and corresponding lesion image features based on the pulmonary hypertension category recognition result, and the lesion image features specifically relate to image features of pulmonary artery dilation, pulmonary parenchyma change, heart shape change or chronic thromboembolism.

[0104] The present application also provides a processing device from the hardware structure, refer to Figure 3 , Figure 3 Fig. 1 shows a schematic diagram of the processing device of the present application. Specifically, the processing device of the present application can include a processor 301, a memory 302 and an input / output device 303. The processor 301 is configured to execute the computer program stored in the memory 302 to realize the functions of the present application, such as Figure 1 the steps of the pulmonary hypertension detection method of fusing image features and tricuspid regurgitation velocity in the corresponding embodiments; or the processor 301 is configured to execute the computer program stored in the memory 302 to realize the functions of the units in the corresponding embodiments, such as Figure 2 the steps of the pulmonary hypertension detection method of fusing image features and tricuspid regurgitation velocity in the corresponding embodiments. Figure 1 the computer program required by the pulmonary hypertension detection method of fusing image features and tricuspid regurgitation velocity in the corresponding embodiments.

[0105] For example, the computer program can be divided into one or more modules / units, one or more modules / units are stored in the memory 302 and executed by the processor 301 to complete the present application. One or more modules / units can be a series of computer program instructions capable of completing a specific function, which is used to describe the execution process of the computer program in the computer device.

[0106] The processing device can include, but is not limited to, the processor 301, the memory 302, the input / output device 303. Those skilled in the art can understand that the schematic diagram is only an example of the processing device and does not constitute a limitation on the processing device, which can include more or less components than the diagram, or combine certain components, or different components, for example, the processing device can also include a network access device, a bus, etc. The processor 301, the memory 302, the input / output device 303 are connected through the bus.

[0107] The processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like. The processor is a control center of the processing device, and connects various parts of the entire device through various interfaces and lines.

[0108] The memory 302 can be used to store computer programs and / or modules, and the processor 301 realizes various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 302, and calling data stored in the memory 302. The memory 302 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function, and the like; and the data storage area can store data created according to the use of the processing device, and the like. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0109] When the processor 301 is used to execute the computer programs stored in the memory 302, the following functions can be realized:

[0110] obtaining a to-be-detected pulmonary computed tomography image and a to-be-detected B-ultrasonic three-tricuspid regurgitation velocity corresponding to a current pulmonary hypertension detection task;

[0111] input the to-be-detected lung computed tomography image and the to-be-detected B-ultrasonic tricuspid regurgitation velocity into a pre-configured pulmonary hypertension detection model, wherein the pulmonary hypertension detection model comprises a tissue site detection model part and a pulmonary hypertension detection model part, the tissue site detection model part is configured to perform classification and recognition of a target tissue site related to a pulmonary hypertension condition on the lung computed tomography image input into the model, and the pulmonary hypertension detection model part is configured to perform pulmonary hypertension detection processing on the classification and recognition result output by the tissue site detection model part in combination with the B-ultrasonic tricuspid regurgitation velocity input into the model, and a pulmonary hypertension detection result obtained through the pulmonary hypertension detection processing is taken as a model output;

[0112] extract the target pulmonary hypertension detection result output by the pulmonary hypertension detection model.

[0113] Those skilled in the art can clearly understand the specific working processes of the pulmonary hypertension detection device fusing the image features and the tricuspid regurgitation velocity, the processing equipment and the corresponding units thereof described above for the convenience and brevity of description, which can be referred to as Figure 1 The description of the pulmonary hypertension detection method fusing the image features and the tricuspid regurgitation velocity in the corresponding embodiments will not be repeated here in detail.

[0114] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions or controlled by instructions related to hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0115] Therefore, the present application provides a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the present application as Figure 1 The steps of the pulmonary hypertension detection method fusing the image features and the tricuspid regurgitation velocity in the corresponding embodiments can be referred to as Figure 1 The description of the pulmonary hypertension detection method fusing the image features and the tricuspid regurgitation velocity in the corresponding embodiments will not be repeated here in detail.

[0116] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0117] Due to the instructions stored in the computer readable storage medium, the present application as Figure 1 The steps of the pulmonary hypertension detection method fusing the image features and the tricuspid regurgitation velocity in the corresponding embodiments, therefore, can implement the present application as Figure 1The beneficial effects that can be achieved by the pulmonary hypertension detection method of fusing image features and tricuspid regurgitation velocity in the corresponding embodiments are described in the foregoing, and will not be described here.

[0118] The pulmonary hypertension detection method of fusing image features and tricuspid regurgitation velocity, the device, the processing equipment and the computer readable storage medium provided by the present application are described in detail above, the principle and implementation mode of the present application are described in this paper, and the above embodiment is only used to help understand the core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description should not be understood as the limitation of the present application.

Claims

1. A method for detecting pulmonary arterial hypertension by fusing image features and tricuspid regurgitation velocity, characterized in that, The method includes: Acquire the computed tomography (CT) image of the lungs to be detected and the tricuspid regurgitation velocity to be detected corresponding to the current pulmonary hypertension detection task; The computed tomography (CT) image of the lung to be detected and the tricuspid regurgitation velocity of the ultrasound to be detected are input into a pre-selected pulmonary hypertension detection model. The pulmonary hypertension detection model includes a tissue site detection model and a pulmonary hypertension detection model. The tissue site detection model is used to classify and identify target tissue sites related to pulmonary hypertension based on the CT image of the lung input to the model. The pulmonary hypertension detection model is used to combine the classification and identification results output by the tissue site detection model with the tricuspid regurgitation velocity of the ultrasound input to the model to perform pulmonary hypertension detection processing. The resulting pulmonary hypertension detection processing result is used as the model output. Extract the target pulmonary hypertension detection results output by the pulmonary hypertension detection model; The tissue site detection model specifically adopts the HydraNet model, and the pulmonary hypertension detection model specifically adopts the multimodal CNN-ViT model. The multimodal CNN-ViT model includes a CNN branch and a ViT branch. The CNN branch specifically uses a ResNet-50 network and is responsible for extracting deep visual features from the classification and recognition results output by the tissue detection model. It also continues to model global dependencies through the ViT branch. Finally, the image representations output by the two branches and the standardized tricuspid regurgitation velocity input to the model via ultrasound are input into a multilayer perceptron for final feature fusion and classification.

2. The method of claim 1, wherein, The target tissue sites specifically include the pulmonary artery, descending aorta, left ventricle, and right ventricle.

3. The method of claim 1, wherein, The method further includes: Obtain computed tomography (CT) images of the lungs of the sample and the corresponding ultrasound tricuspid regurgitation velocity of the sample; The lung computed tomography images of the sample and the tricuspid regurgitation velocity on ultrasound of the sample are assigned corresponding annotations; The pulmonary hypertension detection model was trained using labeled computed tomography images of the lungs of the sample and B-mode ultrasound images of the tricuspid regurgitation velocity of the sample.

4. The method of claim 1, wherein, The operation of the pulmonary hypertension detection model also includes: Before inputting the classification and recognition results output by the tissue site detection model into the pulmonary hypertension detection model, the ROI region image corresponding to the classification and recognition results is subjected to contrast enhancement processing.

5. The method of claim 1, wherein, The method further includes: The results of the target pulmonary hypertension detection are displayed. In addition to the pulmonary hypertension category identification results, the target pulmonary hypertension detection results also include the tricuspid regurgitation velocity to be detected by ultrasound and the corresponding lesion image features. Specifically, the lesion image features involve image features of pulmonary artery dilation, lung parenchymal changes, cardiac morphological changes, or chronic thromboembolism.

6. A pulmonary hypertension detection device integrating imaging features and tricuspid regurgitation velocity, characterized in that, The device includes: The acquisition unit is used to acquire the computed tomography image of the lungs to be detected and the tricuspid regurgitation velocity of the ultrasound corresponding to the current pulmonary hypertension detection task. The detection unit is used to input the lung computed tomography image to be detected and the tricuspid regurgitation velocity on ultrasound to be detected into a pre-selected pulmonary hypertension detection model. The pulmonary hypertension detection model includes a tissue site detection model and a pulmonary hypertension detection model. The tissue site detection model is used to classify and identify target tissue sites related to pulmonary hypertension based on the lung computed tomography image input into the model. The pulmonary hypertension detection model is used to combine the classification and identification results output by the tissue site detection model with the tricuspid regurgitation velocity on ultrasound input into the model to perform pulmonary hypertension detection processing. The resulting pulmonary hypertension detection processing result is used as the model output. The extraction unit is used to extract the target pulmonary hypertension detection result output by the pulmonary hypertension detection model; The tissue site detection model specifically adopts the HydraNet model, and the pulmonary hypertension detection model specifically adopts the multimodal CNN-ViT model. The multimodal CNN-ViT model includes a CNN branch and a ViT branch. The CNN branch specifically uses a ResNet-50 network and is responsible for extracting deep visual features from the classification and recognition results output by the tissue detection model. It also continues to model global dependencies through the ViT branch. Finally, the image representations output by the two branches and the standardized tricuspid regurgitation velocity input to the model via ultrasound are input into a multilayer perceptron for final feature fusion and classification.

7. A processing device, characterized by It includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the method as described in any one of claims 1 to 5 when it invokes the computer program in the memory.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the method of any one of claims 1 to 5.

Images