Ultrasonic image automatic quantitative evaluation method and system based on artificial intelligence

Abstract

The invention discloses an ultrasonic image automatic quantitative evaluation method and system based on artificial intelligence, and relates to the technical field of medical image processing. And respectively acquiring a modal category of the ultrasonic image, a detection frame of each target structure, a structure type and / or a sampling line position and a detection frame of each cardiac cycle through the depth image classification model and the target detection model. And the section type of the ultrasonic image is accurately judged according to the data. According to the method, in combination with the modal category and section type of the ultrasonic image and the image type contained in the ultrasonic image, an appropriate quantitative analysis mode can be accurately provided for different ultrasonic sections, and a quantitative analysis result is visualized, so that a doctor can observe the result conveniently. According to the invention, comprehensive analysis of the multi-modal section can be realized, and more comprehensive and deep analysis and evaluation of the heart structure or function can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to an automatic quantitative evaluation method and system for ultrasonic images based on artificial intelligence. Background Art

[0002] The accurate evaluation of cardiac structure and function is crucial for the diagnosis, treatment, and prognosis of diseases. Echocardiography is a widely used cardiac imaging technique. However, in actual clinical applications, the image quality of echocardiography is affected by various factors. Moreover, the quantitative analysis of echocardiography depends on the experience and subjective judgment of doctors, which affects the accuracy and consistency of diagnostic results. In addition, echocardiography examinations require the measurement and analysis of multiple sections, and the data processing process is time-consuming and prone to errors.

[0003] Applying artificial intelligence technology to the automatic quantitative analysis of echocardiography is expected to overcome the limitations of traditional clinical diagnosis, improve the accuracy and efficiency of diagnosis, and reduce the interference of human factors.

[0004] However, existing automatic quantitative methods mainly perform parameter analysis or functional analysis on single-modal sections, lacking the comprehensive analysis ability of multi-modal sections and making it difficult to achieve a more comprehensive and in-depth analysis and evaluation of cardiac structure or function.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide an automatic quantitative evaluation method and system for ultrasonic images based on artificial intelligence in view of the above-mentioned defects of the existing technology, aiming to solve the problem that existing automatic quantitative methods mainly perform parameter analysis or functional analysis on single-modal sections, lacking the comprehensive analysis ability of multi-modal sections and making it difficult to achieve a more comprehensive and in-depth analysis and evaluation of cardiac structure or function.

[0007] The technical solution adopted by the present invention to solve the problem is as follows:

[0008] In the first aspect, an embodiment of the present invention provides an automatic quantitative evaluation method for ultrasonic images based on artificial intelligence, and the method includes:

[0009] Obtain the medical record data of a patient; the medical record data includes the basic information of the patient, the original ultrasonic image, and the scanning parameters, and the scanning parameters include the ultrasonic physical distance conversion coefficient;

[0010] Preprocess the original ultrasonic image to obtain the ultrasonic image to be analyzed;

[0011] Input the ultrasonic image to be analyzed into a depth image classification model to obtain the ultrasonic modality category; and input the ultrasonic image to be analyzed into an object detection model to obtain the detection box, structure type and / or sampling line position of each target structure and the detection box of each cardiac cycle;

[0012] Determine the ultrasonic section type according to the structure type and quantity of the target structures included in the ultrasonic image to be analyzed and / or the ultrasonic modality category;

[0013] Determine a quantitative analysis method according to at least one piece of information among the ultrasonic section type, the target ultrasonic modality, and the image type included in the ultrasonic image to be analyzed, and visualize the quantitative analysis result; the quantitative analysis method includes at least one of key point positioning and key point distance calculation of cardiac structure measurement items, echocardiogram phase discrimination, blood flow spectrum image analysis, and tissue Doppler spectrum image analysis.

[0014] In a second aspect, an embodiment of the present invention further provides an automatic quantitative evaluation system for ultrasonic images based on artificial intelligence, and the system includes:

[0015] A medical record data management module for obtaining the medical record data of a patient; the medical record data includes the basic information of the patient, the original ultrasonic image, and the scanning parameters, and the scanning parameters include the ultrasonic physical distance conversion coefficient;

[0016] Preprocess the original ultrasonic image to obtain the ultrasonic image to be analyzed;

[0017] An ultrasonic data classification module for inputting the ultrasonic image to be analyzed into a depth image classification model to obtain the ultrasonic modality category; and inputting the ultrasonic image to be analyzed into an object detection model to obtain the detection box, structure type and / or sampling line position of each target structure and the detection box of each cardiac cycle;

[0018] Determine the ultrasonic section type according to the structure type and quantity of the target structures included in the ultrasonic image to be analyzed and / or the ultrasonic modality category;

[0019] An ultrasonic data quantitative analysis module for determining a quantitative analysis method according to at least one piece of information among the ultrasonic section type, the target ultrasonic modality, and the image type included in the ultrasonic image to be analyzed; the quantitative analysis method includes at least one of key point positioning and key point distance calculation of cardiac structure measurement items, echocardiogram phase discrimination, blood flow spectrum image analysis, and tissue Doppler spectrum image analysis;

[0020] An analysis result visualization module for visualizing the quantitative analysis result.

[0021] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which multiple instructions are stored. The instructions are adapted to be loaded and executed by a processor to implement the steps of the artificial intelligence-based automatic quantitative evaluation method for ultrasonic images as described in any one of the above.

[0022] Advantages of the present invention: In the embodiment of the present invention, a depth image classification model and an object detection model are used to respectively obtain the modality category of the ultrasonic image, the detection frame of each target structure, the structure type and / or the sampling line position, and the detection frame of each cardiac cycle. Then, based on these data, the section type of the ultrasonic image can be accurately discriminated. Combining the modality category, section type of the ultrasonic image and the image types it contains, a suitable quantitative analysis method can be accurately provided for different ultrasonic sections, and the quantitative analysis results can be visualized for doctors to observe the results. The present invention can realize the comprehensive analysis of multi-modal sections and achieve a more comprehensive and in-depth analysis and evaluation of cardiac structure or function. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 It is a schematic flowchart of the artificial intelligence-based automatic quantitative evaluation method for ultrasonic images provided by an embodiment of the present invention.

[0025] Figure 2 It is a schematic flowchart of modality classification provided by an embodiment of the present invention.

[0026] Figure 3 It is a schematic flowchart of structure detection provided by an embodiment of the present invention.

[0027] Figure 4 It is a schematic diagram of the cardiac motion curve and the maximum and minimum values provided by an embodiment of the present invention.

[0028] Figure 5 It is a schematic flowchart of spectrum image processing provided by an embodiment of the present invention.

[0029] Figure 6 It is a schematic diagram of the visualization of quantitative analysis results provided by an embodiment of the present invention.

[0030] Figure 7 It is a schematic diagram of the modules of the artificial intelligence-based automatic quantitative evaluation system for ultrasonic images provided by an embodiment of the present invention.

[0031] Figure 8It is a schematic block diagram of the terminal provided by an embodiment of the present invention. Detailed implementation manners

[0032] The present invention discloses an automatic quantitative evaluation method and system for ultrasonic images based on artificial intelligence. To make the objectives, technical solutions and effects of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0033] Aiming at the above defects of the prior art, the present invention provides an automatic quantitative evaluation method for ultrasonic images based on artificial intelligence, as Figure 1 shown. The method specifically includes the following steps:

[0034] Step S100: Obtain the medical record data of the patient; the medical record data includes the basic information of the patient, the original ultrasonic image, and the scanning parameters, and the scanning parameters include the ultrasonic physical distance conversion coefficient.

[0035] Specifically, the medical record data in this embodiment can be from the patient DICOM data folder imported into the system. In practical applications, the system first receives and imports the DICOM format data of the patient. DICOM data contains medical images and their related metadata, such as the basic information of the patient, the original ultrasonic image, and the scanning parameters. The import process involves parsing the DICOM file, identifying the file structure, and preparing the data for subsequent processing. This step is the basis of the entire process, ensuring the accuracy and effectiveness of subsequent information extraction and data conversion.

[0036] After successfully importing the DICOM data, the system will automatically extract the relevant information of the patient. This information may include the basic information of the patient (such as name, age, gender, etc.), the examination type, the examination date, etc. The extraction process may involve parsing the DICOM metadata, aiming to provide the necessary patient background information for subsequent data processing and analysis. Among them, it also involves extracting the ultrasonic physical distance conversion coefficients from the DICOM data, and these coefficients are crucial for converting the pixel spacing in the image data into actual physical measurement values (such as length, area, volume, etc.). The extraction process may require parsing specific tags in the DICOM data, and these tags contain the necessary conversion information. Further, after extracting the data, unit conversion and calibration may be required to ensure the accuracy of the data and ensure the measurement accuracy and consistency of the ultrasonic image.

[0037] Finally, it is necessary to perform format conversion on the DICOM data. The system converts the processed DICOM data into another format, such as the AVI video format or other required formats, making the data more suitable for artificial intelligence analysis and processing, and providing the necessary data support for subsequent artificial intelligence analysis and processing.

[0038] In one implementation, data preprocessing and storage can also be achieved through structured storage technology: use a relational database to store the basic information of patients and scanning parameters. Store the original ultrasound images in the file system in DICOM format, and establish an index table to associate the images with patient IDs. Further, image artifacts (such as acoustic shadows and reverberations) can be automatically detected, low-quality images can be marked, and a reminder for rescan can be triggered.

[0039] Step S200: Preprocess the original ultrasound image to obtain an ultrasound image to be analyzed.

[0040] Specifically, the main purpose of preprocessing in this embodiment is to unify the morphology of the original ultrasound image, so as to facilitate subsequent image processing (such as image segmentation, edge detection, feature extraction, etc.). Through preprocessing, the consistency requirement of the input image size for subsequent artificial intelligence algorithms can be met, thereby improving the accuracy and reliability of image analysis.

[0041] In one implementation, preprocessing the original ultrasound image to obtain an ultrasound image to be analyzed includes:

[0042] Adjust the image size and / or pixel value range of the original ultrasound image to obtain an ultrasound image to be analyzed.

[0043] Specifically, by adjusting the image size, a solid foundation can be laid for subsequent image size unification processing. By adjusting the pixel value range, the pixel value range of the image can be unified to [0,1], which helps to improve the consistency and accuracy of subsequent processing.

[0044] Taking the morphology adjustment of ultrasound images as an example: Since the ultrasound images may have a non-square morphology (i.e., the aspect ratio is not 1:1), a short-side padding operation can be performed to adjust its morphology: calculate the difference between the shorter side and the longer side of the image, and then evenly add black edges (i.e., pixel value is 0) on the corresponding edges of the image (the direction where the short side is located) until the width and height of the image are equal, reaching an aspect ratio of 1:1.

[0045] Regarding the size adjustment of ultrasound images: In order to meet the consistency requirement of the input image size for subsequent artificial intelligence algorithms, an efficient interpolation algorithm can be used to adjust the size of the ultrasound image. For example, bilinear interpolation or bicubic interpolation can be used to scale the ultrasound image after adjusting the aspect ratio, and the ultrasound image can be adjusted to a unified size of 512x512 pixels.

[0046] Regarding the adjustment of the pixel value range of ultrasound images: Since the pixel value ranges of different ultrasound images are inconsistent, the following formula can be used to adjust the pixel value range of ultrasound images:

[0047]

[0048] Thus, the value range of the image pixels is unified to [0, 1].

[0049] Step S300: Input the ultrasound image to be analyzed into a depth image classification model to obtain the ultrasound modality category; and input the ultrasound image to be analyzed into an object detection model to obtain the detection frame, structure type, and / or sampling line position of each target structure and the detection frame of each cardiac cycle.

[0050] Specifically, the depth image classification model is trained with a large number of ultrasound image data of different categories and can learn the feature models of various ultrasound images. Its main function is to judge the modality category of the input ultrasound image. The ultrasound modality refers to different ways of ultrasound imaging. The common modality categories of ultrasound images include but are not limited to: Color, B-mode, blood flow spectrum, tissue Doppler spectrum, and M-mode. After inputting the ultrasound image to be analyzed into the trained depth image classification model, the model will automatically judge which ultrasound modality category it belongs to according to the features presented by the image.

[0051] The object detection model can not only identify the object categories existing in the image but also determine the specific positions of the objects in the image. The objects here refer to specific structures in the ultrasound image, such as ventricles, atria, valves, aorta, etc. The detection frame is a rectangular frame used to enclose the target structure. The object detection model will identify which structure type each enclosed target structure belongs to, which helps to quickly understand the information of different tissues and organs contained in the image. The sampling line is a virtual line used to guide the acquisition of ultrasound signals in a specific direction for subsequent analysis. At different cardiac cycle stages, the structure and position of the heart may change to a certain extent. The object detection model can give a detection frame for each cardiac cycle respectively to more carefully observe and analyze the morphology, motion conditions, etc. of the heart at different times.

[0052] In one implementation, the depth image classification model is used to:

[0053] Extract high-dimensional features of the ultrasound image to be analyzed through depth convolution operations;

[0054] Output a modality probability distribution curve according to the high-dimensional features through fully connected operations;

[0055] Determine the ultrasound modality category of the ultrasound image to be analyzed according to the modality corresponding to the maximum value of the modality probability distribution curve.

[0056] Specifically, in this embodiment, a trained deep image classification model is used for modal classification of ultrasonic images. The deep image classification algorithms involved in the deep image classification model include but are not limited to: ResNet18, MobileNetV3, Vision Transformer. The modal classification of ultrasonic images can provide necessary modal information for subsequent section classification and select the corresponding cardiac structure detection model.

[0057] As Figure 2 shown, the modal classification process of ultrasonic images specifically includes the following steps:

[0058] (1) The original ultrasonic image is first preprocessed to obtain a preprocessed ultrasonic image with an aspect ratio of 1:1 and an image pixel range of [0,1] as the model input image;

[0059] (2) The preprocessed ultrasonic image is input into the deep image classification model, high-dimensional features of the image are extracted through deep convolution operations, and a modal probability distribution curve is output through fully connected operations;

[0060] (3) Determine the abscissa corresponding to the maximum value of the modal probability distribution curve, and the modality corresponding to this abscissa is the ultrasonic modality corresponding to the ultrasonic image.

[0061] In one implementation, the object detection model is used for:

[0062] Extracting high-dimensional features of the ultrasonic image to be analyzed;

[0063] If the ultrasonic image to be analyzed is an echocardiogram, the detection box coordinates and structure types of each target structure in the echocardiogram are predicted according to the high-dimensional features;

[0064] If the ultrasonic image to be analyzed is a spectral ultrasonic image composed of an echocardiogram and a spectral image, the detection box coordinates and structure types of each target structure in the echocardiogram, the sampling line position, and the detection box of each cardiac cycle also need to be predicted according to the high-dimensional features.

[0065] Specifically, in this embodiment, a trained object detection model is used for object recognition of ultrasonic images. In practical applications, advanced object detection models such as yoloV5, Mask R-CNN, Faster R-CNN, etc. can be selected. The ultrasonic image structure detection process involves detecting specific structures from ultrasonic images, such as ventricles, atria, valves, aorta, etc. For spectral ultrasonic data (i.e., including both echocardiograms and spectral images) additional sampling lines and sampling point recognition are added. As Figure 3As shown, the preprocessed ultrasound image is input into the object detection model. The model extracts high-dimensional image features from the image and predicts the coordinates of the detection bounding boxes for each target structure in the echocardiogram and their corresponding types. If the input ultrasound image is an echocardiogram and the modality is B-mode, the types and positions of each cardiac structure are identified, such as the left ventricle, left atrium, right ventricle, right atrium, interventricular septum, etc.; if the input ultrasound image is a spectral ultrasound image, which consists of a two-dimensional image (echocardiogram) and a spectral image, then in addition to identifying the necessary cardiac structures, the sampling line position and the detection bounding boxes for each cardiac cycle also need to be identified. This thus plays a crucial fundamental role in subsequent ultrasound section classification.

[0066] Step S400: Determine the ultrasound section type according to the structural type and quantity of the target structures included in the ultrasound image to be analyzed and / or the ultrasound modality category.

[0067] Specifically, the ultrasound section discrimination process needs to use the types and quantities of the previously identified target structures, as well as the ultrasound modality category as the analysis basis. Specific sections usually display specific quantities and categories of relevant structures, so the types and quantities of the previously identified target structures can assist in judging the ultrasound section type. In addition, different ultrasound modalities focus on providing different aspects of information and can also assist in judging the ultrasound section type.

[0068] For example, the corresponding relationship between the echocardiogram structure detection results and the ultrasound section type is preset in advance, that is, the ultrasound section discrimination rule is determined. In actual applications, the preset ultrasound section discrimination rule and the current echocardiogram structure detection results can be combined for ultrasound section discrimination, thereby enhancing the interpretability of section discrimination and improving the recognition accuracy and precision of sections such as B-mode, blood flow spectrum, and tissue Doppler spectrum. Specifically, set the ultrasound section classification rules as shown in Table 1, and classify the ultrasound sections according to the cardiac structure types and quantities corresponding to different ultrasound sections. For example, when there are a left ventricle, a left atrium, and a right atrium on the ultrasound image and the ultrasound modality is determined to be B-mode, it is determined to be an apical four-chamber view; when the sampling line is placed on the outer wall of the mitral annulus and the ultrasound modality is tissue Doppler spectrum, it is determined to be the tissue Doppler spectrum of the outer wall of the mitral annulus.

[0069] Table 1. Section classification rules

[0070]

[0071]

[0072] Furthermore, after the ultrasound section type determination is completed, section verification can also be performed to improve the reliability of the discrimination result. For example, section verification can be achieved through morphological analysis:

[0073] Calculate whether the spatial relationship between computational structures (such as the angle between the interventricular septum and the mitral valve) conforms to the spatial relationship corresponding to the determined cross-sectional type;

[0074] Alternatively, perform shape matching with a standard cross-sectional template to determine whether it conforms to the shape corresponding to the determined cross-sectional type.

[0075] Step S500: Determine a quantitative analysis method based on at least one of the information including the ultrasonic cross-sectional type, the target ultrasonic modality, and the image type included in the ultrasonic image to be analyzed, and visualize the quantitative analysis result; the quantitative analysis method includes at least one of key point positioning and key point distance calculation of cardiac structure measurement items, echocardiogram phase discrimination, blood flow spectrum image analysis, and tissue Doppler spectrum image analysis.

[0076] Specifically, in this embodiment, different echocardiogram quantitative analysis schemes will be selected according to the ultrasonic cross-sectional classification result in combination with information such as the ultrasonic modality to achieve quantitative evaluation and analysis of cardiac ultrasound structure and function in multiple cross-sections and multiple directions. In practical applications, the subsequent process of the ultrasonic image (which has been preprocessed) is divided into multiple quantitative analysis directions. For example, one quantitative analysis direction is to sequentially perform B-mode key point positioning, cardiac structure key point distance calculation, and echocardiogram phase discrimination; another quantitative analysis direction is spectrum image waveform segmentation model and spectrum parameter measurement.

[0077] In one implementation manner, determining a quantitative analysis method based on at least one of the information including the ultrasonic cross-sectional type, the target ultrasonic modality, and the image type included in the ultrasonic image to be analyzed includes:

[0078] If the ultrasonic modality category belongs to the B modality, the quantitative analysis method includes key point positioning and key point distance calculation of cardiac structure measurement items, and echocardiogram phase discrimination;

[0079] Key point positioning and key point distance calculation of cardiac structure measurement items include: performing key point detection on the cardiac structure measurement items of the ultrasonic image to be analyzed through a depth key point detection model to obtain the key point positions of the cardiac structure measurement items; calculating the Euclidean distance between two endpoints according to the key point positions of the cardiac structure measurement items to obtain the key point distance of the cardiac structure measurement items;

[0080] The calculation method of echocardiogram phase discrimination includes: determining the target calculation method of the cardiac motion curve according to the ultrasonic cross-sectional type; performing full-sequence structure measurement on the echocardiogram according to the target calculation method to obtain the target cardiac motion curve; and determining the echocardiogram phase according to the target cardiac motion curve.

[0081] Specifically, when the ultrasound image is a B-mode image, it is necessary to adopt various pre-set quantitative analysis methods corresponding to the B-mode image, including but not limited to: key point positioning of cardiac structure measurement items, key point distance calculation, echocardiogram phase discrimination, etc.

[0082] For the key point positioning of the cardiac structure: The depth key point positioning models that can be selected include but are not limited to models such as FCN, UNet, DeepLabV3+, and yoloV8. The key point positions of the cardiac structure measurement items to be measured can be determined through the depth key point positioning network. Specifically, the depth key point positioning network will encode the input ultrasound image to extract high-dimensional features such as the edges, textures, and spatial relationships of the cardiac structure, and use the powerful encoding ability of deep learning to further reconstruct and decode these features, and predict information including but not limited to the key point probability distribution map, key point coordinate values, and key point coordinate offset vectors, providing key point coordinate information for the next stage of cardiac structure measurement and visualization.

[0083] For the calculation of the distance between key points of the cardiac structure: Locate the structures at both ends of the cardiac structure measurement item, and use the Euclidean distance formula to calculate the distance between two key points (x1, y1, x2, y2), and multiply by the ultrasonic physical distance conversion coefficient (δ) to convert the pixel distance into the actual physical distance. The overall measurement item calculation formula is as follows:

[0084]

[0085] For the echocardiogram phase discrimination: All measurement parameters related to the B-mode echocardiogram need to be measured at the specified echocardiogram phase (here the echocardiogram phase refers to a specific state in the cardiac systolic and diastolic movements, such as the end-systolic phase and the end-diastolic phase of the heart, etc.). By performing weighted summation on the results of the full-sequence structure measurement of the echocardiogram, a cardiac motion curve is obtained (here weighted summation refers to weighted addition of the cardiac structures related to cardiac motion). The calculation formulas of the cardiac motion curves corresponding to different cross-sectional types are different, and are specifically as follows:

[0086] A4C motion = W1 * LVID l + W2 * LA l + W3 * RA l ;

[0087] PLAX motion = W4 * LV ap + W5 * LA ap + W6 * RVOT Prox ;

[0088] RV-A4C motion = W7 * RVl +W8*RV base-t +W9*RV middle-t ;

[0089] Among them, A4C motion , PLAX motion , RV-A4C motion respectively represent the motion curve of the apical four-chamber view, the cardiac motion curve of the parasternal long-axis view of the left ventricle, and the cardiac motion curve of the apical four-chamber view mainly of the right ventricle; W1 to W9 respectively represent the weights of each measurement item; LVID l is the superior-inferior diameter of the left ventricle, LA l is the superior-inferior diameter of the left atrium, RA l is the superior-inferior diameter of the right atrium; LV ap is the anterior-posterior diameter of the left ventricle, LA ap is the anterior-posterior diameter of the left atrium, RVOT Prox is the internal diameter of the right ventricular outflow tract; RV l is the superior-inferior diameter of the right ventricle, RV base-t is the left-right diameter of the basal segment of the right ventricle, RV middle-t is the left-right diameter of the middle segment of the right ventricle.

[0090] Finally, the corresponding cardiac motion curve calculation formula is determined according to the current echocardiogram section category, and the cardiac motion curve is calculated (as Figure 4 shown). The maximum and minimum values of the cardiac motion curve are found according to the calculated cardiac motion curve, and the echocardiogram phase is determined according to the found maximum and minimum values.

[0091] In one implementation, the quantitative analysis method is determined according to at least one of the ultrasonic section type, the target ultrasonic modality, and the image type included in the ultrasonic image to be analyzed, including:

[0092] If the ultrasonic image to be analyzed includes a blood flow spectrum image, the quantitative analysis method includes blood flow spectrum image analysis;

[0093] The analysis method of blood flow spectrum image analysis includes: segmenting the single-cycle waveform of the blood flow spectrum image through a depth segmentation model to obtain waveform contours corresponding to several waveforms; extracting waveform contour features corresponding to each of the waveform contours; calculating several first quantitative parameters according to each of the waveform contour features and the ultrasonic physical distance conversion coefficient; the first quantitative parameters include at least one of the maximum blood flow velocity, average velocity, and velocity time integral.

[0094] Specifically, as Figure 5As shown, when the ultrasound image contains a blood flow spectrum image, multiple preset quantitative analysis methods corresponding to the blood flow spectrum image need to be adopted, including but not limited to: segmentation of the waveform profile, extraction of waveform profile features, and measurement and calculation of blood flow spectrum parameters.

[0095] Regarding the segmentation of the waveform profile: The depth segmentation models that can be selected include but are not limited to models such as FCN, UNet, DeepLabV3+, and yoloV8. The single-cycle waveform of the spectrum image extracted by the structure detector is segmented through the depth segmentation model to obtain the profile of each waveform. Specifically, after the ultrasound spectrum image containing the blood flow spectrum is input into the depth segmentation model, the depth convolution operation is used to analyze information such as texture, pixels, edges, and key points of the contour in the input image, and high-dimensional features of the input image are extracted. The high-dimensional features of each pixel block are compared with the waveform profile features through the depth feature decoder, and the probability value of each pixel being a waveform is predicted. At the same time, the position of the key points of the contour is regressed to further constrain the shape of the predicted contour. Finally, the mask prediction of each waveform in the spectrum image is realized, that is, the boundary contour of each waveform is obtained, providing a basis for subsequent spectrum quantification.

[0096] Regarding the extraction of waveform profile features and the measurement and calculation of blood flow spectrum parameters: Calculation and analysis are carried out according to the waveform profile, and different waveform features are extracted to meet the measurement requirements of different spectrum parameters. The specific waveform features to be extracted are as follows:

[0097] y max = max{y i |I(x i ,y i ) = 1, i = 1, 2,..., N};

[0098] Δ x = max{x i |I(x i ,y i ) = 1, i = 1, 2,..., N}-min{x i |I(x i ,y i ) = 1, i =

[0099] 1, 2,..., N};

[0100]

[0101] Among them, y max is the waveform peak value, I(x i ,y i ) is the coordinate x i ,y iThe pixel values of the waveform mask at the location, N is the number of non-zero pixels in the mask; Δ x is the width of the waveform, and Area is the waveform area;

[0102] According to the waveform profile characteristics and combined with the ultrasonic physical distance conversion coefficient, calculate various blood flow spectral parameters, including but not limited to quantitative parameters such as the maximum blood flow velocity, average velocity, velocity time integral, etc.

[0103] In one implementation, determine the quantitative analysis method according to at least one piece of information among the ultrasonic section type, the target ultrasonic modality, and the image type included in the ultrasonic image to be analyzed, including:

[0104] If the ultrasonic image to be analyzed contains a tissue Doppler spectral image, the quantitative analysis method includes tissue Doppler spectral image analysis;

[0105] The analysis method of tissue Doppler spectral image analysis includes: performing key point detection on the cardiac cycle waveform of the tissue Doppler spectral image through a depth key point detection model to obtain peak key points corresponding to several cardiac cycle waveforms; calculating several second quantitative parameters according to each of the peak key points and the ultrasonic physical distance conversion coefficient; the second quantitative parameters include at least one of the systolic velocity, early diastolic velocity, and atrial systolic velocity.

[0106] Specifically, as Figure 5 shown, when the ultrasonic image contains a tissue Doppler spectral image, it is necessary to adopt various quantitative analysis methods corresponding to the preset tissue Doppler spectral image, including but not limited to: key point detection, measurement and calculation of tissue Doppler parameters.

[0107] For key point detection: You can choose to use a depth key point detection model based on a heat map, including but not limited to models such as FCN, Unet, DeepLabV3+, yoloV8, etc. Perform key point detection on the cardiac cycle waveform of the spectral image extracted by the structure detector through the depth key point detection model to obtain the peak key points of each waveform. Specifically, after inputting the spectral image into the depth key point detection model, analyze the key point information in the spectral image (which can be in the form of a heat map) through depth convolution operations, and extract high-dimensional features. Predict the probability that each pixel position is a key point through a depth feature decoder, and the pixel position with the maximum probability is the key point position.

[0108] For the measurement and calculation of tissue Doppler parameters: Calculate various tissue Doppler parameters according to the key point position and combined with the ultrasonic physical distance conversion coefficient, including but not limited to: systolic velocity, early diastolic velocity, and atrial systolic velocity.

[0109] The list of echocardiogram measurement parameters is shown in Table 2.

[0110] Table 2 List of echocardiogram measurement parameters

[0111]

[0112]

[0113]

[0114] In one implementation, visualizing the quantitative analysis results includes:

[0115] If the ultrasonic modality category belongs to the B modality, each key point is displayed on the original ultrasonic image using a solid circle, and the key points corresponding to the endpoints of each measurement item are connected by line segments, and different measurement items are visualized using different colors;

[0116] If the ultrasonic image to be analyzed includes a blood flow spectrum image, the waveform profile is drawn on the original blood flow spectrum image;

[0117] If the ultrasonic image to be analyzed includes a tissue Doppler spectrum image, each peak point is drawn on the original ultrasonic image in the form of a solid circle, and a line segment perpendicular to the baseline is drawn starting from the peak point; among them, different peaks are visualized using different colors.

[0118] Specifically, in order to intuitively provide the system prediction results to the doctor for review, it is necessary to perform visual processing on the quantitative analysis results. For B-mode measurement: Each key point is displayed on the original ultrasonic image using a solid circle, and the key points corresponding to the endpoints of the measurement item are linked using line segments. In order to better distinguish different measurement items, a unique color can be assigned to the visualization result of each measurement item. For blood flow spectrum measurement: The waveform profile is drawn on the original blood flow spectrum image. For tissue Doppler spectrum measurement: Each peak point is drawn on the original ultrasonic image in the form of a solid circle, and a line segment perpendicular to the baseline is drawn starting from the peak point, and different colors are used to distinguish between different peaks. This embodiment can provide a good visual effect of the system's automatic quantitative analysis for doctors, improve doctors' trust, and provide the possibility for the system to be put into clinical use. The visual effect is as Figure 6 shown.

[0119] The advantages of the present invention are as follows:

[0120] 1. The present invention not only realizes the classification of echocardiogram modalities and sections, but also realizes the automatic quantitative analysis of different modalities and different ultrasonic sections, greatly simplifies the doctor's work process, and at the same time improves the consistency of the doctor's evaluation of the heart structure and function and reduces subjectivity.

[0121] 2. The present invention realizes the detection of echocardiogram phases and the division of the cardiac cycle, truly achieving fully automated quantitative analysis of echocardiograms and improving the doctor's diagnosis efficiency.

[0122] 3. The echocardiogram quantitative analysis system of the present invention covers a wide range of sections and modalities and has a high degree of automation.

[0123] Based on the above embodiments, the present invention also provides an automatic quantitative evaluation system for ultrasonic images based on artificial intelligence, as Figure 7 shown. The system includes:

[0124] A medical record data management module 01 for obtaining the medical record data of a patient; the medical record data includes the basic information of the patient, the original ultrasonic image, and the scanning parameters, and the scanning parameters include the ultrasonic physical distance conversion coefficient;

[0125] Preprocess the original ultrasonic image to obtain the ultrasonic image to be analyzed;

[0126] An ultrasonic data classification module 02 for inputting the ultrasonic image to be analyzed into a depth image classification model to obtain the ultrasonic modality category; and inputting the ultrasonic image to be analyzed into an object detection model to obtain the detection frame, structure type, and / or sampling line position of each target structure and the detection frame of each cardiac cycle;

[0127] Determine the ultrasonic section type according to the structure type and quantity of the target structures included in the ultrasonic image to be analyzed and / or the ultrasonic modality category;

[0128] An ultrasonic data quantitative analysis module 03 for determining the quantitative analysis method according to at least one of the ultrasonic section type, the target ultrasonic modality, and the image type included in the ultrasonic image to be analyzed; the quantitative analysis method includes at least one of the key point positioning and key point distance calculation of the cardiac structure measurement item, the echocardiogram phase discrimination, the blood flow spectrum image analysis, and the tissue Doppler spectrum image analysis;

[0129] An analysis result visualization module 04 for visualizing the quantitative analysis result.

[0130] Specifically, the four core modules of the system in this embodiment are: a case data management module, an ultrasound data classification module, an ultrasound data quantitative analysis module, and an analysis result visualization module. The case data management module is used to extract the main information and perform data preprocessing on the case ultrasound data imported into the system; the ultrasound data classification module is used to identify the modality and section type of each ultrasound data; the ultrasound data quantitative analysis module is used to perform quantitative analysis on the ultrasound data to obtain the evaluation results of the cardiac structure and function; the analysis result visualization module is used to provide better visualization and interaction functions for doctors.

[0131] Based on the above embodiment, the present invention further provides a terminal, and its principle block diagram can be as Figure 8 shown. The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes an automatic quantitative evaluation method for ultrasound images based on artificial intelligence. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0132] Those skilled in the art can understand that Figure 8 the principle block diagram shown in

[0133] merely shows the block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0134] In summary, the present invention discloses an automatic quantitative evaluation method and system for ultrasonic images based on artificial intelligence, which relates to the technical field of medical image processing. By using a deep image classification model and an object detection model, the modality category of the ultrasonic image, the detection box of each target structure, the structure type and / or the sampling line position, and the detection box of each cardiac cycle are obtained respectively. Furthermore, based on these data, the section type of the ultrasonic image can be accurately discriminated. Combining the modality category, the section type of the ultrasonic image and the image types it contains, a suitable quantitative analysis method can be accurately provided for different ultrasonic sections, and the quantitative analysis results are visualized for doctors to observe the results. The present invention can realize the comprehensive analysis of multi-modal sections and achieve a more comprehensive and in-depth analysis and evaluation of cardiac structure or function.

[0135] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. An automatic quantitative evaluation method for ultrasonic images based on artificial intelligence, characterized in that, The method includes: Obtaining the medical record data of a patient; the medical record data includes the patient's basic information, original ultrasound images, and scanning parameters, and the scanning parameters include an ultrasound physical distance conversion coefficient; Preprocessing the original ultrasound images to obtain the ultrasound images to be analyzed; Inputting the ultrasound images to be analyzed into a depth image classification model to obtain the ultrasound modality categories; and inputting the ultrasound images to be analyzed into an object detection model to obtain the detection frames, structure types, and / or sampling line positions of each target structure and the detection frames of each cardiac cycle; Determining the ultrasound section type according to the structure types and quantities of the target structures included in the ultrasound images to be analyzed and / or the ultrasound modality categories; Determining a quantitative analysis method according to at least one of the ultrasound section type, the target ultrasound modality, and the image type included in the ultrasound images to be analyzed, and visualizing the quantitative analysis results; the quantitative analysis method includes at least one of key point positioning and key point distance calculation for cardiac structure measurement items, echocardiogram phase discrimination, blood flow spectral image analysis, and tissue Doppler spectral image analysis.

2. The automatic quantitative evaluation method of ultrasonic images based on artificial intelligence according to claim 1, characterized in that Preprocessing the original ultrasound images to obtain the ultrasound images to be analyzed, including: Adjusting the image size and / or pixel value range of the original ultrasound images to obtain the ultrasound images to be analyzed.

3. The automatic quantitative evaluation method of ultrasonic images based on artificial intelligence according to claim 1, wherein The depth image classification model is used for: Extracting high-dimensional features of the ultrasound images to be analyzed through depth convolution operations; Outputting a modality probability distribution curve according to the high-dimensional features through fully connected operations; Determining the ultrasound modality category of the ultrasound images to be analyzed according to the modality corresponding to the maximum value of the modality probability distribution curve.

4. The automatic quantitative evaluation method of ultrasonic images based on artificial intelligence according to claim 1, characterized in that The object detection model is used for: Extracting high-dimensional features of the ultrasound images to be analyzed; If the ultrasound images to be analyzed are echocardiograms, predicting the detection frame coordinates and structure types of each target structure in the echocardiograms according to the high-dimensional features; If the ultrasound images to be analyzed are spectral ultrasound images composed of echocardiograms and spectral images, predicting the detection frame coordinates and structure types, sampling line positions of each target structure in the echocardiograms, and the detection frames of each cardiac cycle according to the high-dimensional features.

5. The method for automatically quantitatively evaluating ultrasonic images based on artificial intelligence according to claim 1, wherein Determining a quantitative analysis method according to at least one of the ultrasound section type, the target ultrasound modality, and the image type included in the ultrasound images to be analyzed, including: If the ultrasound modality category belongs to the B modality, the quantitative analysis method includes key point positioning and key point distance calculation for cardiac structure measurement items, and echocardiogram phase discrimination; Key point positioning and key point distance calculation for cardiac structure measurement items include: detecting key points of the cardiac structure measurement items of the ultrasound images to be analyzed through a depth key point detection model to obtain the key point positions of the cardiac structure measurement items; calculating the Euclidean distance between two endpoints according to the key point positions of the cardiac structure measurement items to obtain the key point distances of the cardiac structure measurement items. The calculation method for echocardiogram phase discrimination includes: determining the target calculation method for the cardiac motion curve according to the type of the ultrasonic section; performing full-sequence structure measurement on the echocardiogram according to the target calculation method to obtain the target cardiac motion curve; and determining the echocardiogram phase according to the target cardiac motion curve.

6. The automatic quantitative evaluation method for ultrasonic images based on artificial intelligence according to claim 1, characterized in that, Determining the quantitative analysis method according to at least one piece of information among the type of the ultrasonic section, the target ultrasonic modality, and the type of the ultrasonic image to be analyzed, including: If the ultrasonic image to be analyzed includes a blood flow spectrum image, the quantitative analysis method includes blood flow spectrum image analysis; The analysis method of blood flow spectrum image analysis includes: segmenting the single-cycle waveform of the blood flow spectrum image through a depth segmentation model to obtain waveform contours corresponding to several waveforms; extracting waveform contour features corresponding to each of the waveform contours; and calculating several first quantitative parameters according to each of the waveform contour features and the ultrasonic physical distance conversion coefficient; the first quantitative parameters include at least one parameter among the maximum blood flow velocity, the average velocity, and the velocity time integral.

7. The automatic quantitative evaluation method of ultrasonic images based on artificial intelligence according to claim 1, wherein Determining the quantitative analysis method according to at least one piece of information among the type of the ultrasonic section, the target ultrasonic modality, and the type of the ultrasonic image to be analyzed, including: If the ultrasonic image to be analyzed includes a tissue Doppler spectrum image, the quantitative analysis method includes tissue Doppler spectrum image analysis; The analysis method of tissue Doppler spectrum image analysis includes: detecting key points of the cardiac cycle waveform of the tissue Doppler spectrum image through a depth key point detection model to obtain peak key points corresponding to several cardiac cycle waveforms; and calculating several second quantitative parameters according to each of the peak key points and the ultrasonic physical distance conversion coefficient; the second quantitative parameters include at least one parameter among the systolic velocity, the early diastolic velocity, and the atrial systolic velocity.

8. The automatic quantitative evaluation method for ultrasonic images based on artificial intelligence according to claim 1, characterized in that Visualizing the quantitative analysis result, including: If the ultrasonic modality category belongs to the B modality, displaying each key point on the original ultrasonic image by using a solid circle, connecting the key points corresponding to the endpoints of each measurement item through a line segment, and visualizing different measurement items by using different colors; If the ultrasonic image to be analyzed includes a blood flow spectrum image, drawing the waveform contour on the original blood flow spectrum image; If the ultrasonic image to be analyzed includes a tissue Doppler spectrum image, drawing each peak point in the form of a solid circle on the original ultrasonic image, and drawing a line segment perpendicular to the baseline starting from the peak point; wherein, different peaks are visualized by using different colors.

9. An automatic quantitative evaluation system for ultrasonic images based on artificial intelligence, characterized in that, The system includes: A medical record data management module, configured to obtain the medical record data of a patient; the medical record data includes the basic information of the patient, the original ultrasonic image, and the scanning parameters, and the scanning parameters include the ultrasonic physical distance conversion coefficient; Preprocessing the original ultrasonic image to obtain the ultrasonic image to be analyzed; An ultrasonic data classification module, configured to input the ultrasonic image to be analyzed into a depth image classification model to obtain an ultrasonic modality category; and input the ultrasonic image to be analyzed into an object detection model to obtain a detection box, a structure type and / or a sampling line position of each target structure, and a detection box of each cardiac cycle; Determine an ultrasonic section type according to the structure type and quantity of the target structure included in the ultrasonic image to be analyzed and / or the ultrasonic modality category; An ultrasonic data quantitative analysis module, configured to determine a quantitative analysis method according to at least one piece of information among the ultrasonic section type, the target ultrasonic modality, and the image type included in the ultrasonic image to be analyzed; the quantitative analysis method includes at least one of key point positioning and key point distance calculation of cardiac structure measurement items, echocardiogram phase discrimination, blood flow spectrum image analysis, and tissue Doppler spectrum image analysis; An analysis result visualization module, configured to visualize the quantitative analysis result.

10. A computer-readable storage medium having a plurality of instructions stored thereon, characterized in that, The instruction is applicable to be loaded and executed by a processor to implement the steps of the method for automatically quantitatively evaluating an ultrasonic image based on artificial intelligence according to any one of claims 1-8.

Images