A processing method and device for classifying cardiac ultrasound images

Abstract

Embodiments of the present invention relate to a method and apparatus for classifying cardiac ultrasound images. The method comprises: receiving a cardiac ultrasound image as a first ultrasound image; classifying the ultrasound technology type and slice type of the first ultrasound image based on an M-mode classification model to generate a first classification vector; classifying the image quality level and slice integrity level of the first ultrasound image based on a quality detection model to generate a second classification vector; and classifying the image data of the first ultrasound image based on the first and second classification vectors. The present invention can improve the classification efficiency of cardiac ultrasound images.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a processing method and device for classifying cardiac ultrasonic images. Background Art

[0002] Echocardiograms (also known as cardiac ultrasound images) can display the internal cross-sectional structures of the heart, such as the ventricles, atria, and arteries. When managing cardiac ultrasound image data, data classification is required according to specific requirements. These classifications can be broadly classified into the following: by ultrasound technique, by section, by image quality, and by section integrity. Ultrasound technique classification can be categorized into various ultrasound technique types, including two-dimensional echocardiography, spectral Doppler echocardiography, and color Doppler echocardiography. Section classification can be categorized into various view types, including the parasternal long axis, apical four-chamber view, parasternal left ventricular short axis, and subxiphoid view. Image quality classification can be categorized into various image quality levels, such as levels 1, 2, 3, and so on, with progressive quality enhancement. Section integrity classification can be categorized into various sections, such as levels 1, 2, 3, and so on, with progressive quality enhancement. Under normal circumstances, the above-mentioned classification tasks are all achieved through manual classification. Affected by human factors, this conventional processing method can neither output high classification efficiency nor guarantee stable classification quality. Summary of the Invention

[0003] The present invention addresses the shortcomings of existing technologies by providing a method, device, electronic device, and computer-readable storage medium for classifying cardiac ultrasound images. Two classification models (an M-type classification model and a quality detection model) are provided. The M-type classification model classifies cardiac ultrasound images by ultrasound technique and section type, while the quality detection model assesses the image quality and section integrity of cardiac ultrasound images. This invention improves the efficiency of cardiac ultrasound image classification while ensuring stable classification quality.

[0004] To achieve the above-mentioned object, a first aspect of an embodiment of the present invention provides a method for classifying cardiac ultrasound images, the method comprising:

[0005] receiving a cardiac ultrasound image as a corresponding first ultrasound image;

[0006] Based on a preset M-type classification model, the ultrasound technology type and the section type of the first ultrasound image are classified to generate a corresponding first classification vector; the number of ultrasound technology types in the M-type classification model is a first number D1, the number of section types in the M-type classification model is a second number D2, and the number M is the total number of model categories in the M-type classification model, D1 ≥ 1, D2 ≥ 1, and M = D1 * D2 ≥ 1; the first classification vector includes the first classification probabilities of the number M, each of the first classification probabilities corresponding to a type of ultrasound technology type and a type of section type;

[0007] The image quality level and the slice integrity level of the first ultrasound image are classified based on a preset quality detection model to generate a corresponding second classification vector; the number of image quality levels of the quality detection model is a third number D3, and the number of slice integrity levels of the quality detection model is a fourth number D4; the second classification vector includes a first classification sub-vector and a second classification sub-vector; the first classification sub-vector includes first-level probabilities of the third number D3, each first-level probability corresponding to an image quality level; the second classification sub-vector includes second-level probabilities of the fourth number D4, each second-level probability corresponding to a slice integrity level;

[0008] Image data classification processing is performed on the first ultrasound image according to the first and second classification vectors.

[0009] Preferably, the M-type classification model includes a first feature extraction module, an ultrasound classification module, a section classification module, and a first classification fusion module; the output end of the first feature extraction module is connected to the input end of the ultrasound classification module and the section classification module respectively; the input end of the first classification fusion module is connected to the output end of the ultrasound classification module and the section classification module respectively; the first feature extraction module is a ResNet-50 network; the ultrasound classification module and the section classification module are both deep learning classification networks;

[0010] The first feature extraction module is used to perform feature extraction processing on the input ultrasound image to generate a corresponding first feature map;

[0011] The ultrasound classification module is used to perform ultrasound technology type classification processing on the input first feature map to generate a corresponding first ultrasound technology classification vector; the first ultrasound technology classification vector includes the first number D1 of first ultrasound technology classification probabilities;

[0012] The section classification module is used to perform section type classification processing on the input first feature map to generate a corresponding first section classification vector; the first section classification vector includes the first section classification probability of the second quantity D2;

[0013] The first classification fusion module is used to arrange and combine the first ultrasound technology classification probability of the input first quantity D1 and the first section classification probability of the second quantity D2 to obtain the first probability combination of the quantity M, each first probability combination consisting of one first ultrasound technology classification probability and one first section classification probability; and perform weighted sum calculation on the first ultrasound technology classification probability and the first section classification probability of each first probability combination to obtain the corresponding first probability; and perform normalization processing on all the first probabilities to obtain the classification vector output corresponding to the normalized probability composition of the quantity M.

[0014] Preferably, before classifying the ultrasound technology type and section type of the first ultrasound image based on a preset M-mode classification model, the method further includes:

[0015] Step 31, initialize the first training counter to 0;

[0016] Step 32: Obtain any two training cardiac ultrasound images from a preset training database as corresponding first training images and second training images; each of the training cardiac ultrasound images corresponds to a first ultrasound technology type label and a first section type label;

[0017] Step 33: performing a truth tensor setting based on the first ultrasound technology type label and the first slice type label of the first training image to generate a corresponding first truth vector; and performing a truth tensor setting based on the first ultrasound technology type label and the first slice type label of the second training image to generate a corresponding second truth vector;

[0018] Step 34: Input the first training image into the M-mode classification model to classify the ultrasound technique type and the section type to obtain a corresponding first training classification vector; and input the second training image into the M-mode classification model to classify the ultrasound technique type and the section type to obtain a corresponding second training classification vector;

[0019] Step 35: Identify whether the first section type labels of the first and second training images are the same section type; if so, substitute the first and second training classification vectors and the first and second true value vectors into a preset first-class model loss function L a Calculate the corresponding first loss value; if not, substitute the first and second training classification vectors and the first and second true value vectors into the preset two-class model loss function L b Calculating and obtaining the corresponding first loss value;

[0020] Step 36, identify whether the first loss value satisfies the preset first loss value convergence range; when the first loss value satisfies the first loss value convergence range, add 1 to the first training counter, and identify whether the new first training counter exceeds the preset first training times threshold, if so, stop model training, otherwise go to step 32 to continue training; when the first loss value does not satisfy the first loss value convergence range, modulate the model parameters and go to step 34 to continue training when the modulation is completed.

[0021] Furthermore, the first truth value vector is recorded as the first truth value vector The first truth vector The first true value probability of the quantity M is included 1≤i≤M; each of the first true value probabilities A type of section type corresponding to a type of ultrasound technology; the first truth value vector There is only one first true value probability The probability value of the first true value is 1, the rest are 0, and the probability value is 1 The corresponding ultrasound technology type and section type match the corresponding first ultrasound technology type label and the first section type label;

[0022] The second truth value vector is recorded as the second truth value vector The second truth vector The second true value probability of the quantity M is included The second true value probability of each A type of section type corresponding to a type of ultrasound technology; the second truth value vector There is only one second true value probability The probability value of the second true value is 1, the rest are 0, and the probability value is 1 The corresponding ultrasound technology type and section type match the corresponding first ultrasound technology type label and the first section type label;

[0023] The first training classification vector is recorded as the first training classification vector Y1; the first training classification vector Y1 includes the first training probability y of the number M. 1,i ; Each of the first training probabilities y 1,i A type of section corresponding to a type of ultrasound technology;

[0024] The second training classification vector is recorded as the second training classification vector Y2; the second training classification vector Y2 includes the second training probability y of the number M 2,i Each of the second training probabilities y 2,iA type of section corresponding to a type of ultrasound technology;

[0025] The loss function L of the model a for:

[0026]

[0027] in,

[0028] The first multi-class cross entropy loss function L CE,1 for:

[0029] The second multi-classification cross entropy loss function L CE,2 for:

[0030] Distance loss function L S Including L1 loss function, L2 loss function, Smooth L1 loss function and Huber loss function;

[0031] The two-class model loss function L b for:

[0032]

[0033] Preferably, the classifying the ultrasound technology type and the section type of the first ultrasound image based on a preset M-mode classification model to generate a corresponding first classification vector specifically includes:

[0034] The first feature extraction module of the M-type classification model performs feature extraction processing on the input first ultrasound image to generate the corresponding first feature map and sends it to the ultrasound classification module and the section classification module;

[0035] The ultrasound classification module performs ultrasound technology type classification processing on the first feature map to generate a corresponding first ultrasound technology classification vector and sends it to the first classification fusion module; the first ultrasound technology classification vector includes the first number D1 of the first ultrasound technology classification probabilities;

[0036] The section classification module performs section type classification processing on the first feature map to generate the corresponding first section classification vector and sends it to the first classification fusion module; the first section classification vector includes the second number D2 of the first section classification probabilities;

[0037] The first classification fusion module arranges and combines the first ultrasound technology classification probabilities of the first quantity D1 and the first section classification probabilities of the second quantity D2 to obtain the first probability combinations of the quantity M, and each first probability combination consists of one first ultrasound technology classification probability and one first section classification probability; and performs weighted summation on the first ultrasound technology classification probabilities and the first section classification probabilities of each first probability combination to obtain the corresponding first probability; and performs normalization processing on all the first probabilities to obtain the corresponding first normalized probability; and uses each first normalized probability as the corresponding first classification probability; and the obtained first classification probabilities of the quantity M constitute the corresponding first classification vector output.

[0038] Preferably, the quality detection model includes a second feature extraction module, an image quality classification module, a section integrity classification module, and a second classification fusion module; the output end of the second feature extraction module is connected to the input end of the image quality classification module and the section integrity classification module respectively; the input end of the second classification fusion module is connected to the output end of the image quality classification module and the section integrity classification module respectively; the second feature extraction module is a convolutional neural network; the image quality classification module and the section integrity classification module are both deep learning regression networks;

[0039] The second feature extraction module is used to perform feature extraction processing on the input ultrasound image to generate a corresponding second feature map;

[0040] The image quality classification module is used to perform image quality classification processing on the input second feature map to generate a corresponding first quality level classification vector; the first quality level classification vector includes the first quality level classification probability of the third quantity D3;

[0041] The slice integrity classification module is used to perform slice integrity level classification processing on the input second feature map to generate a corresponding first integrity level classification vector; the first slice classification vector includes the first integrity level classification probability of the fourth quantity D4;

[0042] The second classification fusion module is configured to take the input first quality level classification vector and the first integrity level classification vector as corresponding classification sub-vectors and compose a corresponding classification vector from the obtained two classification sub-vectors to output.

[0043] Preferably, before classifying the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model, the method further includes:

[0044] Step 71, initialize the second training counter to 0;

[0045] Step 72: Obtain any one training ultrasound cardiac image from a preset training database as a corresponding third training image; each of the training ultrasound cardiac images corresponds to a first image quality level label;

[0046] Step 73: Perform truth value tensor setting according to the first image quality level label of the third training image to generate a corresponding third truth value vector;

[0047] Step 74: Input the third training image into the second feature extraction module of the quality detection model to perform feature extraction to generate the corresponding second feature map; and input the second feature map into the image quality classification module to perform image quality grade classification processing to obtain the corresponding third training classification vector;

[0048] Step 75: Substitute the third training classification vector and the third true value vector into a preset first regression loss function to calculate and obtain a corresponding second loss value;

[0049] Step 76, identify whether the second loss value satisfies the preset second loss value convergence range; when the second loss value satisfies the second loss value convergence range, add 1 to the second training counter, and identify whether the new second training counter exceeds the preset second training times threshold, if so, stop the model training of the second feature extraction module and the image quality classification module, otherwise go to step 72 to continue training; when the second loss value does not satisfy the second loss value convergence range, modulate the model parameters of the second feature extraction module and the image quality classification module and go to step 74 to continue training when the modulation is completed.

[0050] Furthermore, the third truth vector is recorded as the third truth vector The third truth vector The third true value probability of the third quantity D3 1≤j≤D3; each of the third true value probabilities Corresponding to an image quality level; the third true value vector There is only one third true value probability The probability value of the third truth value is 1, the rest are 0, and the probability value is 1 The corresponding image quality level matches the corresponding first image quality level label;

[0051] The third training classification vector is recorded as the third training classification vector Y3; the third training classification vector Y3 includes the third training probability y of the third quantity D33,j Each of the third training probabilities y 3,j Corresponds to an image quality level;

[0052] The first regression loss function includes MSE Loss loss function, MAE Loss loss function and Pseudo-Huber Loss loss function.

[0053] Preferably, before classifying the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model, the method further includes:

[0054] Step 91: Screening cardiac ultrasound images for training of various slice types for various ultrasound technology types from a preset training database to obtain a first number D1 of first-category image sets; each first-category image set corresponds to an ultrasound technology type; each first-category image set includes a plurality of the training cardiac ultrasound images; and each of the training cardiac ultrasound images corresponds to a first slice integrity level label;

[0055] Step 92: Initialize the first set index to 1, initialize the first image number threshold to a preset third training number threshold, and initialize the first training state flag to a single set training flag; the first training state flag includes a single set training flag and a full set training flag;

[0056] Step 93: Set the third training counter to 0; and use the first type of image set corresponding to the first set index as the corresponding current image set;

[0057] Step 94: Select any one of the training cardiac ultrasound images from the current image set as a corresponding fourth training image; and perform truth value tensor setting based on the first slice integrity level label of the fourth training image to generate a corresponding fourth truth value vector;

[0058] Step 95: Input the fourth training image into the second feature extraction module of the quality inspection model to perform feature extraction to generate the corresponding second feature map; input the second feature map into the section integrity classification module to perform section integrity level classification processing to obtain a corresponding fourth training classification vector; and substitute the fourth training classification vector and the fourth true value vector into a preset second regression loss function to calculate and obtain a corresponding third loss value;

[0059] Step 96: Identify whether the third loss value satisfies a preset third loss value convergence range. When the third loss value satisfies the third loss value convergence range, increment the third training counter by 1, and identify whether the new third training counter exceeds the first image quantity threshold. If so, proceed to step 97; otherwise, proceed to step 94 to continue training. When the second loss value does not satisfy the third loss value convergence range, modulate the model parameters of the second feature extraction module and the section integrity classification module, and when the modulation is complete, proceed to step 95 to continue training.

[0060] Step 97, identifying the first training state flag; if the first training state flag is a single set training flag, adding 1 to the first set index and going to step 98; if the first training state flag is a full set training flag, going to step 99;

[0061] Step 98: Identify whether the first set index is greater than the first number D1. If the first set index is less than or equal to the first number D1, proceed to step 93 to continue training. If the first set index is greater than the first number D1, merge the first number D1 of first-category image sets into a complete image set as the new current image set, reset the first image number threshold to the number of training cardiac ultrasound images in the current image set, reset the third training counter to 0, set the first training status flag to the full set training flag, and proceed to step 94 to continue training.

[0062] Step 99: Stop model training of the second feature extraction module and the section integrity classification module.

[0063] Furthermore, the fourth truth value vector is recorded as the fourth truth value vector The fourth truth vector The fourth true value probability of the fourth quantity D4 is included 1≤k≤D4; each of the fourth truth value probabilities Corresponding to a section completeness level; the fourth truth vector There is only one fourth true value probability The probability value of the fourth true value y is 1, the rest are 0, and the probability value is 1 The corresponding section integrity level matches the corresponding first section integrity level label;

[0064] The fourth training classification vector is recorded as the fourth training classification vector Y4; the fourth training classification vector Y4 includes the fourth training probability y of the fourth quantity D4 4,k Each of the fourth training probabilities y4,k Corresponding to a section integrity level;

[0065] The second regression loss function includes an MSE Loss loss function, an MAE Loss loss function, and a Pseudo-Huber Loss loss function.

[0066] Preferably, the classifying the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model to generate a corresponding second classification vector specifically includes:

[0067] The second feature extraction module of the quality detection model performs feature extraction processing on the input first ultrasound image to generate a corresponding second feature map and sends it to the image quality classification module and the section integrity classification module;

[0068] The image quality classification module performs image quality classification processing on the second feature map to generate a corresponding first quality level classification vector and sends it to the second classification fusion module; the first quality level classification vector includes the third number D3 of the first quality level classification probabilities;

[0069] The slice integrity classification module performs slice integrity level classification processing on the second feature map to generate a corresponding first integrity level classification vector and sends it to the second classification fusion module; the first slice classification vector includes the first integrity level classification probability of the fourth number D4;

[0070] The second classification fusion module takes each first quality level classification probability of the first quality level classification vector as the corresponding first level probability, and the first level probabilities of the third quantity D3 obtained are used to form the corresponding first classification sub-vector; and takes each first integrity level classification probability of the first integrity level classification vector as the corresponding second level probability, and the second level probabilities of the fourth quantity D4 obtained are used to form the corresponding second classification sub-vector; and the first and second sub-vectors obtained are used to form the corresponding second classification vector output.

[0071] Preferably, the performing image data classification processing on the first ultrasound image according to the first and second classification vectors specifically includes:

[0072] The ultrasound technology type and section type corresponding to the first classification probability with the largest probability value in the first classification vector are used as the corresponding first ultrasound technology type label and first section type label; and the image quality level corresponding to the first level probability with the largest probability in the first classification sub-vector of the second classification vector is used as the corresponding first image quality level label; and the section integrity level corresponding to the second level probability with the largest probability in the second classification sub-vector of the second classification vector is used as the corresponding first section integrity level label;

[0073] The first ultrasound image and the corresponding cardiac ultrasound image data record consisting of the first ultrasound technology type label, the first section type label, the first image quality level label and the first section integrity level label are added to a preset cardiac ultrasound image classification database; and the first ultrasound image whose first image quality level label exceeds a preset image quality level threshold and whose first section integrity level exceeds a preset section integrity level threshold is used as a training cardiac ultrasound image, and the training cardiac ultrasound image and the corresponding training data record consisting of the first ultrasound technology type label, the first section type label, the first image quality level label and the first section integrity level label are stored in a preset training database; both the cardiac ultrasound image classification database and the training database support cardiac ultrasound image query functions based on ultrasound technology type, section type, image quality level or section integrity level.

[0074] A second aspect of the embodiments of the present invention provides an apparatus for implementing the method described in the first aspect, comprising: a receiving module, an M-type classification module, a quality detection module, and a data classification module;

[0075] The receiving module is configured to receive a cardiac ultrasound image as a corresponding first ultrasound image;

[0076] The M-type classification module is used to classify the ultrasound technology type and the section type of the first ultrasound image based on a preset M-type classification model to generate a corresponding first classification vector; the number of ultrasound technology types in the M-type classification model is a first number D1, the number of section types in the M-type classification model is a second number D2, and the number M is the total number of model categories of the M-type classification model, D1 ≥ 1, D2 ≥ 1, M = D1 * D2 ≥ 1; the first classification vector includes the first classification probabilities of the number M, each of the first classification probabilities corresponding to a type of ultrasound technology type and a type of section type;

[0077] The quality detection module is configured to classify the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model to generate a corresponding second classification vector; the number of image quality levels of the quality detection model is a third number D3, and the number of slice integrity levels of the quality detection model is a fourth number D4; the second classification vector includes a first classification sub-vector and a second classification sub-vector; the first classification sub-vector includes first-level probabilities of the third number D3, each first-level probability corresponding to an image quality level; the second classification sub-vector includes second-level probabilities of the fourth number D4, each second-level probability corresponding to a slice integrity level;

[0078] The data classification module is used to perform image data classification processing on the first ultrasound image according to the first and second classification vectors.

[0079] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0080] The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method described in the first aspect above;

[0081] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0082] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed by a computer, the computer executes the method described in the first aspect above.

[0083] Embodiments of the present invention provide a processing method, apparatus, electronic device, and computer-readable storage medium for classifying cardiac ultrasound images. These methods provide two classification models: an M-type classification model and a quality detection model. The M-type classification model classifies cardiac ultrasound images by ultrasound technique and section type, while the quality detection model assesses the image quality and section integrity of cardiac ultrasound images. This method improves the classification efficiency of cardiac ultrasound images while ensuring stable classification quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 A schematic diagram of a processing method for classifying cardiac ultrasound images provided in the first embodiment of the present invention;

[0085] Figure 2a This is a module structure diagram of the M-type classification model provided in Example 1 of the present invention;

[0086] Figure 2bThis is a module structure diagram of the quality detection model provided in Example 1 of the present invention;

[0087] Figure 3 A module structure diagram of a processing device for classifying cardiac ultrasound images provided in a second embodiment of the present invention;

[0088] Figure 4 This is a structural diagram of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0089] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0090] A first embodiment of the present invention provides a method for classifying cardiac ultrasound images, such as Figure 1 A schematic diagram of a processing method for classifying cardiac ultrasound images provided in the first embodiment of the present invention is shown. The method mainly includes the following steps:

[0091] Step 1: Receive a cardiac ultrasound image as a corresponding first ultrasound image.

[0092] In this embodiment of the present invention, after obtaining a first cardiac ultrasound image, the first ultrasound image is classified using the M-mode classification model in step 2 based on ultrasound technology and slice type. Furthermore, in step 3 based on quality detection, the image quality and slice integrity of the first ultrasound image are assessed using the quality detection model. To better understand these two models, the M-mode classification model and quality detection model are described below.

[0093] (1) M-type classification model

[0094] like Figure 2a As shown in the module structure diagram of the M-type classification model provided in Example 1 of the present invention, the M-type classification model includes a first feature extraction module, an ultrasound classification module, a section classification module and a first classification fusion module; the output end of the first feature extraction module is connected to the input end of the ultrasound classification module and the section classification module, respectively; the input end of the first classification fusion module is connected to the output end of the ultrasound classification module and the section classification module, respectively; among them, the first feature extraction module defaults to the ResNet-50 network; the ultrasound classification module and the section classification module are both deep learning classification networks.

[0095] The first feature extraction module is used to perform feature extraction processing on the input ultrasound image to generate a corresponding first feature map.

[0096] The ultrasound classification module is configured to classify the input first feature map by ultrasound technology type to generate a corresponding first ultrasound technology classification vector; the first ultrasound technology classification vector includes a first number D1 of first ultrasound technology classification probabilities. Here, the ultrasound technology types in the first embodiment of the present invention include at least two-dimensional echocardiography (cross-sect iona lechocardiography) technology type, spectral Doppler echocardiography (spectral Doppler echocardiography) technology type, and color Doppler echocardiography (color Doppler echocardiography) technology type.

[0097] The section classification module is configured to perform section type classification processing on the input first feature map to generate a corresponding first section classification vector; the first section classification vector includes a second quantity D2 of first section classification probabilities. Here, the section types in the first embodiment of the present invention include at least a parasternal long axis view, an apical four-chamber view, a parasternal left ventricular short axis view, and a subxiphoid view.

[0098] The first classification fusion module is used to arrange and combine the first ultrasound technology classification probability of the input first quantity D1 and the first section classification probability of the second quantity D2 to obtain a first probability combination of quantity M, M = D1*D2, each first probability combination consists of a first ultrasound technology classification probability and a first section classification probability; and perform weighted sum calculation on the first ultrasound technology classification probability and the first section classification probability of each first probability combination to obtain the corresponding first probability; and perform normalization processing on all first probabilities to obtain the classification vector output corresponding to the normalized probability composition of quantity M.

[0099] It should be noted that the M-mode classification model needs to be trained before it is used; that is, before the method of the first embodiment of the present invention classifies the ultrasound technology type and the section type of the first ultrasound image based on the preset M-mode classification model in step 2, the method further includes:

[0100] Step A1, initializing a first training counter to 0;

[0101] Step A2, obtaining any two cardiac ultrasound images for training from a preset training database as corresponding first training images and second training images;

[0102] Each training cardiac ultrasound image corresponds to a first ultrasound technology type label and a first section type label;

[0103] Step A3: performing a truth tensor setting based on the first ultrasound technology type label and the first section type label of the first training image to generate a corresponding first truth vector; and performing a truth tensor setting based on the first ultrasound technology type label and the first section type label of the second training image to generate a corresponding second truth vector;

[0104] Among them, the first truth value vector is recorded as the first truth value vector First truth vector The first true value probability including the quantity M 1≤i≤M; each first true value probability A type of slice type corresponding to a type of ultrasound technology; the first truth vector There is only one first true value probability The probability value of is 1, the rest are 0, and the probability of the first true value is 1 The corresponding ultrasound technology type and section type match the corresponding first ultrasound technology type label and first section type label;

[0105] The second truth vector is denoted as the second truth vector Second truth vector The second true value probability including the quantity M Each second true value probability A type of slice type corresponding to a type of ultrasound technology; the second truth vector There is only one second true value probability The probability value of is 1, the rest are 0, and the second true value probability of probability value is 1 The corresponding ultrasound technology type and section type match the corresponding first ultrasound technology type label and first section type label;

[0106] Step A4: Input the first training image into the M-type classification model to classify the ultrasound technology type and the section type to obtain a corresponding first training classification vector; and input the second training image into the M-type classification model to classify the ultrasound technology type and the section type to obtain a corresponding second training classification vector;

[0107] The first training classification vector is recorded as the first training classification vector Y1; the first training classification vector Y1 includes the first training probability y of the number M 1,i ; Each first training probability y 1,i A type of section corresponding to a type of ultrasound technology;

[0108] The second training classification vector is recorded as the second training classification vector Y2; the second training classification vector Y2 includes the second training probability y of the number M. 2,i ; Each second training probability y 2,iA type of section corresponding to a type of ultrasound technology;

[0109] Step A5: Identify whether the first section type labels of the first and second training images are the same section type; if so, substitute the first and second training classification vectors and the first and second true value vectors into the preset first-class model loss function L a Calculate the corresponding first loss value; if not, substitute the first and second training classification vectors and the first and second true value vectors into the preset two-class model loss function L b Calculate and obtain the corresponding first loss value;

[0110] in,

[0111] A type of model loss function L a for:

[0112]

[0113] The first multi-class cross entropy loss function L CE,1 for:

[0114]

[0115] The second multi-classification cross entropy loss function L CE,2 for:

[0116]

[0117] Distance loss function L S Including L1 loss function, L2 loss function, Smooth L1 loss function and Huber loss function;

[0118] The loss function L of the second-class model b for:

[0119]

[0120] Here, if the section types of the first and second training images are different during model training, a two-class model loss function L consisting of two multi-class (M classification) cross entropy loss functions is used. b Calculate the loss value; if the section types of the first and second training images are the same, overfitting may occur during the model training process. To avoid this, the first embodiment of the present invention specifically uses the second type model loss function L b Add a distance loss function L on the basis of S Constitute a class of model loss function L a , the distance loss function L S It is used to calculate the loss of the distance between two training classification vectors, namely the first and second training classification vectors. The distance loss function LS There are many implementation methods, including L1 loss function, L2 loss function, Smooth L1 loss function and Huber loss function;

[0121] Step A6, identify whether the first loss value meets the preset first loss value convergence range; when the first loss value meets the first loss value convergence range, add 1 to the first training counter, and identify whether the new first training counter exceeds the preset first training times threshold, if so, stop model training, otherwise go to step A2 to continue training; when the first loss value does not meet the first loss value convergence range, modulate the model parameters and go to step A4 to continue training when the modulation is completed.

[0122] Here, the convergence range of the first loss value is a pre-set range of values ​​used to constrain the first loss value. Once the first loss value enters this range, it is considered that the training of the current two training images, namely the first and second training images, is successful; the first training times threshold is a pre-set parameter used to constrain the total number of successful training times. Each time the two training images obtained are successfully trained, the first training counter is increased by 1. If the first training counter after addition does not exceed the first training times threshold, it means that a new pair of training images needs to be obtained for training; if it exceeds the first training times threshold, it means that the current model training can be stopped.

[0123] (2) Quality inspection model

[0124] like Figure 2b As shown in the module structure diagram of the quality detection model provided in Example 1 of the present invention, the quality detection model includes a second feature extraction module, an image quality classification module, a section integrity classification module and a second classification fusion module; the output end of the second feature extraction module is connected to the input end of the image quality classification module and the section integrity classification module, respectively; the input end of the second classification fusion module is connected to the output end of the image quality classification module and the section integrity classification module, respectively; wherein, the second feature extraction module is a convolutional neural network; the image quality classification module and the section integrity classification module are both deep learning regression networks.

[0125] The second feature extraction module is used to perform feature extraction processing on the input ultrasound image to generate a corresponding second feature map.

[0126] The image quality classification module is used to perform image quality classification processing on the input second feature map to generate a corresponding first quality level classification vector; the first quality level classification vector includes a third quantity D3 of first quality level classification probabilities.

[0127] The slice integrity classification module is used to perform slice integrity level classification processing on the input second feature map to generate a corresponding first integrity level classification vector; the first slice classification vector includes a first integrity level classification probability of a fourth quantity D4.

[0128] The second classification fusion module is used to take the input first quality level classification vector and the first integrity level classification vector as respective corresponding classification sub-vectors and compose a corresponding classification vector from the obtained two classification sub-vectors to output.

[0129] It should be noted that the second feature extraction module + the image quality classification module need to be trained before using the quality detection model; that is, before classifying the image quality level and the slice integrity level of the first ultrasound image based on the preset quality detection model in step 3, the method of embodiment 1 of the present invention further includes:

[0130] Step B1, initializing the second training counter to 0;

[0131] Step B2, obtaining any one training cardiac ultrasound image from a preset training database as a corresponding third training image;

[0132] Each training cardiac ultrasound image corresponds to a first image quality level label;

[0133] Step B3, performing truth value tensor setting according to the first image quality level label of the third training image to generate a corresponding third truth value vector;

[0134] Among them, the third truth vector is recorded as the third truth vector The third truth vector The third true value probability including the third quantity D3 1≤j≤D3; each third truth value probability Corresponding to an image quality level; the third truth vector There is only one third true value probability The probability value of is 1, the rest are 0, and the third true value probability of probability value 1 The corresponding image quality level is matched with the corresponding first image quality level label;

[0135] Step B4: Input the third training image into the second feature extraction module of the quality detection model to perform feature extraction to generate a corresponding second feature map; and input the second feature map into the image quality classification module to perform image quality grade classification processing to obtain a corresponding third training classification vector;

[0136] The third training classification vector is recorded as the third training classification vector Y3; the third training classification vector Y3 includes the third training probability y of the third quantity D3 3,j; Each third training probability y 3,j Corresponds to an image quality level;

[0137] Step B5: Substitute the third training classification vector and the third true value vector into a preset first regression loss function to calculate and obtain a corresponding second loss value;

[0138] Among them, the first regression loss function includes MSE Loss loss function, MAE Loss loss function and Pseudo-Huber Loss loss function;

[0139] Here, because the image quality classification module is actually a deep learning regression network, the first embodiment of the present invention uses the regression loss function applicable to the deep learning regression network as the first regression loss function for training the second feature extraction module + the image quality classification module; the first regression loss function of the first embodiment of the present invention supports multiple implementation methods of regression loss functions, including MSE Loss loss function, MAE Loss loss function, and Pseudo-Huber Loss loss function;

[0140] Step B6, identify whether the second loss value meets the preset second loss value convergence range; when the second loss value meets the second loss value convergence range, add 1 to the second training counter, and identify whether the new second training counter exceeds the preset second training times threshold. If so, stop the model training of the second feature extraction module and the image quality classification module; otherwise, go to step B2 to continue training; when the second loss value does not meet the second loss value convergence range, modulate the model parameters of the second feature extraction module and the image quality classification module and go to step B4 to continue training when the modulation is completed.

[0141] Here, the convergence range of the second loss value is a pre-set range for constraining the value range of the second loss value. Once the second loss value enters this range, it is deemed that the training of the current training image, i.e., the third training image, is successful. The second training times threshold is a pre-set parameter for constraining the total number of successful training times. Each time the third training image is successfully trained, the second training counter is increased by 1. If the second training counter after increasing by 1 does not exceed the second training times threshold, it means that new training images need to be obtained for training. If it exceeds the second training times threshold, it means that the training of the second feature extraction module + image quality classification module can be stopped.

[0142] It should be noted that the second feature extraction module + the slice integrity classification module need to be trained before using the quality detection model; that is, before classifying the image quality level and the slice integrity level of the first ultrasound image based on the preset quality detection model in step 3, the method of embodiment 1 of the present invention further includes:

[0143] Step C1, screening cardiac ultrasound images of various sections for training various ultrasound techniques from a preset training database to obtain a first number D1 of first-category image sets;

[0144] Each first-category image set corresponds to an ultrasound technology type; each first-category image set includes a plurality of training cardiac ultrasound images; each training cardiac ultrasound image corresponds to a first section integrity level label;

[0145] Step C2: Initialize the first set index to 1, initialize the first image quantity threshold to a preset third training number threshold, and initialize the first training state flag to a single set training flag;

[0146] The first training status flag includes a single-set training flag and a full-set training flag;

[0147] Step C3, setting the third training counter to 0; and taking the first type of image set corresponding to the first set index as the corresponding current image set;

[0148] Step C4: Selecting a training cardiac ultrasound image from the current image set as the corresponding fourth training image; and performing truth value tensor setting based on the first slice integrity level label of the fourth training image to generate a corresponding fourth truth value vector;

[0149] Among them, the fourth truth value vector is recorded as the fourth truth value vector Fourth truth vector The fourth truth probability including the fourth quantity D4 1≤k≤D4; each fourth truth value probability Corresponding to a level of section completeness; the fourth truth vector There is only one fourth true value probability The probability value of is 1, the rest are 0, and the fourth true value probability of probability value 1 The corresponding section integrity level matches the corresponding first section integrity level label;

[0150] Step C5: Input the fourth training image into the second feature extraction module of the quality detection model to perform feature extraction to generate a corresponding second feature map; input the second feature map into the section integrity classification module to perform section integrity level classification processing to obtain a corresponding fourth training classification vector; and substitute the fourth training classification vector and the fourth true value vector into a preset second regression loss function to calculate a corresponding third loss value;

[0151] The fourth training classification vector is recorded as the fourth training classification vector Y4; the fourth training classification vector Y4 includes the fourth training probability y of the fourth quantity D4 4,k; Each fourth training probability y 4,k Corresponding to a slice completeness level; the second regression loss function includes MSE Loss loss function, MAE Loss loss function and Pseudo-Huber Loss loss function;

[0152] Here, because the section integrity classification module is actually a deep learning regression network, the first embodiment of the present invention is based on the regression loss function applicable to the deep learning regression network as the second regression loss function for training the second feature extraction module + the section integrity classification module; the second regression loss function of the first embodiment of the present invention supports multiple implementation methods of regression loss functions, including MSE Loss loss function, MAE Loss loss function and Pseudo-HuberLoss loss function;

[0153] Step C6: Identify whether the third loss value satisfies a preset third loss value convergence range. When the third loss value satisfies the third loss value convergence range, increment the third training counter by 1, and determine whether the new third training counter exceeds the first image quantity threshold. If so, proceed to step C7; otherwise, proceed to step C4 to continue training. When the second loss value does not satisfy the third loss value convergence range, modulate the model parameters of the second feature extraction module and the section integrity classification module, and when the modulation is complete, proceed to step C5 to continue training.

[0154] Step C7, identifying the first training state flag; if the first training state flag is a single set training flag, adding 1 to the first set index and going to step C8; if the first training state flag is a full set training flag, going to step C9;

[0155] Step C8: Identify whether the first set index is greater than the first number D1; if the first set index is less than or equal to the first number D1, proceed to step C3 to continue training; if the first set index is greater than the first number D1, merge the first number D1 of first-category image sets into a complete image set as a new current image set, reset the first image number threshold to the number of cardiac ultrasound images used for training in the current image set, reset the third training counter to 0, set the first training status flag to the full set training flag, and proceed to step C4 to continue training;

[0156] Step C9: stop the model training of the second feature extraction module and the section integrity classification module.

[0157] Here, it is not difficult to see from the above steps C1-C9 that when training the second feature extraction module + section integrity classification module, the first embodiment of the present invention first uses part or all of the images of an ultrasound technology type image set for training, and then uses part or all of the images of the next ultrasound technology type image set for training, until the first number D1 image sets supported by the M-type classification model of the first embodiment of the present invention are successfully trained, and then all the previously used ultrasound technology type image sets are merged to form a complete image set consisting of D1 first-category image sets, and then based on all the images of the complete image set, a round of training is performed on the second feature extraction module + section integrity classification module. When all the images of the complete image set are successfully trained, the current training of the second feature extraction module + section integrity classification module is stopped.

[0158] Step 2: Classify the ultrasound technology type and the section type of the first ultrasound image based on a preset M-mode classification model to generate a corresponding first classification vector;

[0159] The number of ultrasound technology types in the M-type classification model is a first number D1, the number of section types in the M-type classification model is a second number D2, and the number M is the total number of model categories in the M-type classification model, D1 ≥ 1, D2 ≥ 1, M = D1 * D2 ≥ 1; the first classification vector includes the number M of first classification probabilities, each first classification probability corresponding to a type of ultrasound technology type and a type of section type;

[0160] Specifically, the step 21 includes: a first feature extraction module of the M-type classification model performs feature extraction processing on the input first ultrasound image to generate a corresponding first feature map and sends it to the ultrasound classification module and the section classification module;

[0161] Step 22: The ultrasound classification module performs ultrasound technology type classification processing on the first feature map to generate a corresponding first ultrasound technology classification vector and sends it to the first classification fusion module;

[0162] The first ultrasound technology classification vector includes a first number D1 of first ultrasound technology classification probabilities;

[0163] Step 23: The slice classification module performs slice type classification processing on the first feature map to generate a corresponding first slice classification vector and sends it to the first classification fusion module;

[0164] The first section classification vector includes the first section classification probability of the second quantity D2;

[0165] In step 24, the first classification fusion module arranges and combines the first ultrasound technology classification probabilities of the first quantity D1 and the first section classification probabilities of the second quantity D2 to obtain a quantity M of first probability combinations, where each first probability combination consists of a first ultrasound technology classification probability and a first section classification probability; and performs weighted summation on the first ultrasound technology classification probabilities and the first section classification probabilities of each first probability combination to obtain a corresponding first probability; and performs normalization processing on all first probabilities to obtain a corresponding first normalized probability; and uses each first normalized probability as a corresponding first classification probability; and the obtained quantity M of first classification probabilities constitutes a corresponding first classification vector output.

[0166] Step 3: Classify the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model to generate a corresponding second classification vector;

[0167] The number of image quality levels of the quality detection model is a third number D3, and the number of slice integrity levels of the quality detection model is a fourth number D4; the second classification vector includes a first classification sub-vector and a second classification sub-vector; the first classification sub-vector includes first-level probabilities of the third number D3, each first-level probability corresponding to an image quality level; the second classification sub-vector includes second-level probabilities of the fourth number D4, each second-level probability corresponding to a slice integrity level;

[0168] Specifically, the step 31 includes: a second feature extraction module of the quality detection model performs feature extraction processing on the input first ultrasound image to generate a corresponding second feature map, which is sent to the image quality classification module and the section integrity classification module;

[0169] Step 32: The image quality classification module performs image quality classification processing on the second feature map to generate a corresponding first quality classification vector and sends it to the second classification fusion module;

[0170] wherein the first quality level classification vector includes a third quantity D3 of first quality level classification probabilities;

[0171] Step 33: The slice integrity classification module performs slice integrity level classification processing on the second feature map to generate a corresponding first integrity level classification vector and sends it to the second classification fusion module;

[0172] The first section classification vector includes a first completeness level classification probability of a fourth quantity D4;

[0173] In step 34, the second classification fusion module uses each first quality level classification probability of the first quality level classification vector as the corresponding first level probability, and forms the corresponding first classification sub-vector with the first level probability of the third quantity D3 obtained; and uses each first integrity level classification probability of the first integrity level classification vector as the corresponding second level probability, and forms the corresponding second classification sub-vector with the second level probability of the fourth quantity D4 obtained; and outputs the corresponding second classification vector with the first and second sub-vectors obtained.

[0174] Step 4: performing image data classification processing on the first ultrasound image according to the first and second classification vectors;

[0175] Specifically, the step 41 includes: using the ultrasound technology type and section type corresponding to the first classification probability with the largest probability value in the first classification vector as the corresponding first ultrasound technology type label and first section type label; using the image quality level corresponding to the first level probability with the largest probability in the first classification subvector of the second classification vector as the corresponding first image quality level label; and using the section integrity level corresponding to the second level probability with the largest probability in the second classification subvector of the second classification vector as the corresponding first section integrity level label;

[0176] Step 42: Add the first ultrasound image and the corresponding first ultrasound technology type label, first slice type label, first image quality grade label, and first slice integrity grade label, which constitute a corresponding cardiac ultrasound image data record, to a preset cardiac ultrasound image classification database; and use the first ultrasound image whose first image quality grade label exceeds a preset image quality grade threshold and whose first slice integrity grade exceeds a preset slice integrity grade threshold as a training cardiac ultrasound image, and store the training cardiac ultrasound image and the corresponding first ultrasound technology type label, first slice type label, first image quality grade label, and first slice integrity grade label, which constitute a corresponding training data record, in a preset training database;

[0177] Among them, the cardiac ultrasound image classification database and training database both support cardiac ultrasound image query functions based on ultrasound technology type, section type, image quality level or section integrity level.

[0178] In summary, through the above steps 1-4, the first embodiment of the present invention can continuously analyze massive cardiac ultrasound images without any human intervention, and obtain ultrasound technology type labels, section type labels, image quality level labels, and section integrity level labels corresponding to the images. It can not only meet multiple classification requirements and ensure stable classification quality, but also improve classification efficiency; in addition, the first embodiment of the present invention can also store the obtained classification labels together with the cardiac ultrasound images as training data records in the training database, thereby achieving the purpose of continuously updating the training database and improving the richness of training data.

[0179] Figure 3 This is a module structure diagram of a processing device for classifying cardiac ultrasound images provided in the second embodiment of the present invention. The device can be a terminal device or server that implements the method of the embodiment of the present invention, or a device that implements the method of the embodiment of the present invention connected to the above terminal device or server. For example, the device can be a device or chip system of the above terminal device or server. Figure 3 As shown, the device includes: a receiving module 201, an M-type classification module 202, a quality detection module 203 and a data classification module 204.

[0180] The receiving module 201 is configured to receive a cardiac ultrasound image as a corresponding first ultrasound image.

[0181] The M-type classification module 202 is used to classify the ultrasound technology type and section type of the first ultrasound image based on a preset M-type classification model to generate a corresponding first classification vector; the number of ultrasound technology types of the M-type classification model is a first number D1, the number of section types of the M-type classification model is a second number D2, the number M is the total number of model categories of the M-type classification model, D1≥1, D2≥1, M=D1*D2≥1; the first classification vector includes a number M of first classification probabilities, and each first classification probability corresponds to a type of ultrasound technology type and a type of section type.

[0182] The quality detection module 203 is used to classify the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model to generate a corresponding second classification vector; the number of image quality levels of the quality detection model is a third number D3, and the number of slice integrity levels of the quality detection model is a fourth number D4; the second classification vector includes a first classification sub-vector and a second classification sub-vector; the first classification sub-vector includes a first-level probability of the third number D3, and each first-level probability corresponds to an image quality level; the second classification sub-vector includes a second-level probability of the fourth number D4, and each second-level probability corresponds to a slice integrity level.

[0183] The data classification module 204 is configured to perform image data classification processing on the first ultrasound image according to the first and second classification vectors.

[0184] An embodiment of the present invention provides a processing device for classifying cardiac ultrasound images, which can execute the method steps in the above method embodiment. Its implementation principles and technical effects are similar and will not be repeated here.

[0185] It should be noted that the division of the modules of the above devices is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity or physically separated. Furthermore, these modules can all be implemented in the form of software called by a processing element; or all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the receiving module can be a separate processing element, or it can be integrated into a chip of the above device. In addition, it can be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above-mentioned module. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element or by instructions in the form of software.

[0186] For example, the above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a certain module is implemented in the form of a processing element scheduling program code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0187] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in accordance with the embodiment of the present invention is generated in whole or in part. The above-mentioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) mode. The above-mentioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, DVDs), or semiconductor media (for example, solid-state drives (SSDs)).

[0188] Figure 4 This is a schematic diagram of the structure of an electronic device provided in the third embodiment of the present invention. The electronic device may be the aforementioned terminal device or server, or may be a terminal device or server connected to the aforementioned terminal device or server to implement the method of the embodiment of the present invention. Figure 4 As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303's transceiver actions. Various instructions may be stored in the memory 302 for performing various processing functions and implementing the methods and processing procedures provided in the above embodiments of the present invention. Preferably, the electronic device involved in the embodiment of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The above-mentioned communication port 306 is used for connecting and communicating between the electronic device and other peripheral devices.

[0189] exist Figure 4The system bus mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory (Non-Volume ile Memory), such as at least one disk storage.

[0190] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0191] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, which stores instructions. When the storage medium is run on a computer, it enables the computer to execute the methods and processing procedures provided in the above embodiments.

[0192] An embodiment of the present invention further provides a chip for executing instructions, which is used to execute the methods and processing procedures provided in the above embodiments.

[0193] Embodiments of the present invention provide a processing method, apparatus, electronic device, and computer-readable storage medium for classifying cardiac ultrasound images. These methods provide two classification models: an M-type classification model and a quality detection model. The M-type classification model classifies cardiac ultrasound images by ultrasound technique and section type, while the quality detection model assesses the image quality and section integrity of cardiac ultrasound images. This method improves the classification efficiency of cardiac ultrasound images while ensuring stable classification quality.

[0194] Professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0195] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0196] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for classifying cardiac ultrasound images, characterized in that: The method comprises: receiving a cardiac ultrasound image as a corresponding first ultrasound image; Based on a preset M-type classification model, the ultrasound technology type and the section type of the first ultrasound image are classified to generate a corresponding first classification vector; the number of ultrasound technology types in the M-type classification model is a first number D1, the number of section types in the M-type classification model is a second number D2, and the number M is the total number of model categories in the M-type classification model, D1 ≥ 1, D2 ≥ 1, and M = D1 * D2 ≥ 1; the first classification vector includes the first classification probabilities of the number M, each of the first classification probabilities corresponding to a type of ultrasound technology type and a type of section type; The image quality level and the slice integrity level of the first ultrasound image are classified based on a preset quality detection model to generate a corresponding second classification vector; the number of image quality levels of the quality detection model is a third number D3, and the number of slice integrity levels of the quality detection model is a fourth number D4; the second classification vector includes a first classification sub-vector and a second classification sub-vector; the first classification sub-vector includes first-level probabilities of the third number D3, each first-level probability corresponding to an image quality level; the second classification sub-vector includes second-level probabilities of the fourth number D4, each second-level probability corresponding to a slice integrity level; performing image data classification processing on the first ultrasound image according to the first and second classification vectors; The performing image data classification processing on the first ultrasound image according to the first and second classification vectors specifically includes: The ultrasound technology type and section type corresponding to the first classification probability with the largest probability value in the first classification vector are used as the corresponding first ultrasound technology type label and first section type label; and the image quality level corresponding to the first level probability with the largest probability in the first classification sub-vector of the second classification vector is used as the corresponding first image quality level label; and the section integrity level corresponding to the second level probability with the largest probability in the second classification sub-vector of the second classification vector is used as the corresponding first section integrity level label; The first ultrasound image and the corresponding cardiac ultrasound image data record consisting of the first ultrasound technology type label, the first section type label, the first image quality level label and the first section integrity level label are added to a preset cardiac ultrasound image classification database; and the first ultrasound image whose first image quality level label exceeds a preset image quality level threshold and whose first section integrity level exceeds a preset section integrity level threshold is used as a training cardiac ultrasound image, and the training cardiac ultrasound image and the corresponding training data record consisting of the first ultrasound technology type label, the first section type label, the first image quality level label and the first section integrity level label are stored in a preset training database; both the cardiac ultrasound image classification database and the training database support cardiac ultrasound image query functions based on ultrasound technology type, section type, image quality level or section integrity level.

2. The method for classifying cardiac ultrasound images according to claim 1, wherein: The M-type classification model includes a first feature extraction module, an ultrasound classification module, a section classification module, and a first classification fusion module; the output end of the first feature extraction module is connected to the input end of the ultrasound classification module and the section classification module respectively; the input end of the first classification fusion module is connected to the output end of the ultrasound classification module and the section classification module respectively; the first feature extraction module is a ResNet-50 network; the ultrasound classification module and the section classification module are both deep learning classification networks; The first feature extraction module is used to perform feature extraction processing on the input ultrasound image to generate a corresponding first feature map; The ultrasound classification module is used to perform ultrasound technology type classification processing on the input first feature map to generate a corresponding first ultrasound technology classification vector; the first ultrasound technology classification vector includes the first number D1 of first ultrasound technology classification probabilities; The section classification module is used to perform section type classification processing on the input first feature map to generate a corresponding first section classification vector; the first section classification vector includes the first section classification probability of the second quantity D2; The first classification fusion module is used to arrange and combine the first ultrasound technology classification probability of the input first quantity D1 and the first section classification probability of the second quantity D2 to obtain the first probability combination of the quantity M, each first probability combination consisting of one first ultrasound technology classification probability and one first section classification probability; and perform weighted sum calculation on the first ultrasound technology classification probability and the first section classification probability of each first probability combination to obtain the corresponding first probability; and perform normalization processing on all the first probabilities to obtain the classification vector output corresponding to the normalized probability composition of the quantity M.

3. The method for classifying cardiac ultrasound images according to claim 2, wherein: Before classifying the ultrasound technology type and section type of the first ultrasound image based on the preset M-mode classification model, the method further includes: Step 31, initialize the first training counter to 0; Step 32: Obtain any two training cardiac ultrasound images from a preset training database as corresponding first training images and second training images; each of the training cardiac ultrasound images corresponds to a first ultrasound technology type label and a first section type label; Step 33: performing a truth tensor setting based on the first ultrasound technology type label and the first slice type label of the first training image to generate a corresponding first truth vector; and performing a truth tensor setting based on the first ultrasound technology type label and the first slice type label of the second training image to generate a corresponding second truth vector; Step 34: Input the first training image into the M-mode classification model to classify the ultrasound technique type and the section type to obtain a corresponding first training classification vector; and input the second training image into the M-mode classification model to classify the ultrasound technique type and the section type to obtain a corresponding second training classification vector; Step 35: Identify whether the first section type labels of the first and second training images are the same section type; if so, substitute the first and second training classification vectors and the first and second true value vectors into a preset first-class model loss function L a Calculate the corresponding first loss value; if not, substitute the first and second training classification vectors and the first and second true value vectors into the preset two-class model loss function L b Calculating and obtaining the corresponding first loss value; Step 36, identify whether the first loss value satisfies the preset first loss value convergence range; when the first loss value satisfies the first loss value convergence range, add 1 to the first training counter, and identify whether the new first training counter exceeds the preset first training times threshold, if so, stop model training, otherwise go to step 32 to continue training; when the first loss value does not satisfy the first loss value convergence range, modulate the model parameters and go to step 34 to continue training when the modulation is completed.

4. The method for classifying cardiac ultrasound images according to claim 3, wherein: The first truth value vector is recorded as the first truth value vector The first truth vector The first true value probability of the quantity M is included 1≤i≤M; each of the first true value probabilities A type of section type corresponding to a type of ultrasound technology; the first truth value vector There is only one first true value probability The probability value of the first true value is 1, the rest are 0, and the probability value is 1 The corresponding ultrasound technology type and section type match the corresponding first ultrasound technology type label and the first section type label; The second truth value vector is recorded as the second truth value vector The second truth vector The second true value probability of the quantity M is included The second true value probability of each A type of section type corresponding to a type of ultrasound technology; the second truth value vector There is only one second true value probability The probability value of the second true value is 1, the rest are 0, and the probability value is 1 The corresponding ultrasound technology type and section type match the corresponding first ultrasound technology type label and the first section type label; The first training classification vector is recorded as the first training classification vector Y1; the first training classification vector Y1 includes the first training probability y of the number M. 1,i ; Each of the first training probabilities y 1,i A type of section corresponding to a type of ultrasound technology; The second training classification vector is recorded as the second training classification vector Y2; the second training classification vector Y2 includes the second training probability y of the number M 2,i Each of the second training probabilities y 2,i A type of section corresponding to a type of ultrasound technology; The loss function L of the model a for: in, The first multi-class cross entropy loss function L CE,1 for: The second multi-classification cross entropy loss function L CE,2 for: Distance loss function L S Including L1 loss function, L2 loss function, Smooth L1 loss function and Huber loss function; The two-class model loss function L b for:

5. The method for classifying cardiac ultrasound images according to claim 2, wherein: The classifying the ultrasound technology type and the section type of the first ultrasound image based on the preset M-mode classification model to generate a corresponding first classification vector specifically includes: The first feature extraction module of the M-type classification model performs feature extraction processing on the input first ultrasound image to generate the corresponding first feature map and sends it to the ultrasound classification module and the section classification module; The ultrasound classification module performs ultrasound technology type classification processing on the first feature map to generate a corresponding first ultrasound technology classification vector and sends it to the first classification fusion module; the first ultrasound technology classification vector includes the first number D1 of the first ultrasound technology classification probabilities; The section classification module performs section type classification processing on the first feature map to generate the corresponding first section classification vector and sends it to the first classification fusion module; the first section classification vector includes the second number D2 of the first section classification probabilities; The first classification fusion module arranges and combines the first ultrasound technology classification probabilities of the first quantity D1 and the first section classification probabilities of the second quantity D2 to obtain the first probability combinations of the quantity M, and each first probability combination consists of one first ultrasound technology classification probability and one first section classification probability; and performs weighted summation on the first ultrasound technology classification probabilities and the first section classification probabilities of each first probability combination to obtain the corresponding first probability; and performs normalization processing on all the first probabilities to obtain the corresponding first normalized probability; and uses each first normalized probability as the corresponding first classification probability; and the obtained first classification probabilities of the quantity M constitute the corresponding first classification vector output.

6. The method for classifying cardiac ultrasound images according to claim 1, wherein: The quality detection model includes a second feature extraction module, an image quality classification module, a section integrity classification module, and a second classification fusion module; the output end of the second feature extraction module is connected to the input end of the image quality classification module and the section integrity classification module respectively; the input end of the second classification fusion module is connected to the output end of the image quality classification module and the section integrity classification module respectively; the second feature extraction module is a convolutional neural network; the image quality classification module and the section integrity classification module are both deep learning regression networks; The second feature extraction module is used to perform feature extraction processing on the input ultrasound image to generate a corresponding second feature map; The image quality classification module is used to perform image quality classification processing on the input second feature map to generate a corresponding first quality level classification vector; the first quality level classification vector includes the first quality level classification probability of the third quantity D3; The slice integrity classification module is used to perform slice integrity level classification processing on the input second feature map to generate a corresponding first integrity level classification vector; the first slice classification vector includes the first integrity level classification probability of the fourth quantity D4; The second classification fusion module is configured to take the input first quality level classification vector and the first integrity level classification vector as corresponding classification sub-vectors and compose a corresponding classification vector from the obtained two classification sub-vectors to output.

7. The method for classifying cardiac ultrasound images according to claim 6, wherein: Before classifying the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model, the method further includes: Step 71, initialize the second training counter to 0; Step 72: Obtain any one training ultrasound cardiac image from a preset training database as a corresponding third training image; each of the training ultrasound cardiac images corresponds to a first image quality level label; Step 73: Perform truth value tensor setting according to the first image quality level label of the third training image to generate a corresponding third truth value vector; Step 74: Input the third training image into the second feature extraction module of the quality detection model to perform feature extraction to generate the corresponding second feature map; and input the second feature map into the image quality classification module to perform image quality grade classification processing to obtain the corresponding third training classification vector; Step 75: Substitute the third training classification vector and the third true value vector into a preset first regression loss function to calculate and obtain a corresponding second loss value; Step 76, identify whether the second loss value satisfies the preset second loss value convergence range; when the second loss value satisfies the second loss value convergence range, add 1 to the second training counter, and identify whether the new second training counter exceeds the preset second training times threshold, if so, stop the model training of the second feature extraction module and the image quality classification module, otherwise go to step 72 to continue training; when the second loss value does not satisfy the second loss value convergence range, modulate the model parameters of the second feature extraction module and the image quality classification module and go to step 74 to continue training when the modulation is completed.

8. The method for classifying cardiac ultrasound images according to claim 7, wherein: The third truth value vector is recorded as the third truth value vector The third truth vector The third true value probability of the third quantity D3 1≤j≤D3; each of the third true value probabilities Corresponding to an image quality level; the third true value vector There is only one third true value probability The probability value of the third truth value is 1, the rest are 0, and the probability value is 1 The corresponding image quality level matches the corresponding first image quality level label; The third training classification vector is recorded as the third training classification vector Y3; the third training classification vector Y3 includes the third training probability y of the third quantity D3 3,j Each of the third training probabilities y 3,j Corresponds to an image quality level; The first regression loss function includes MSE Loss loss function, MAE Loss loss function and Pseudo-HuberLoss loss function.

9. The method for classifying cardiac ultrasound images according to claim 6, wherein: Before classifying the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model, the method further includes: Step 91: Screening cardiac ultrasound images for training of various slice types for various ultrasound technology types from a preset training database to obtain a first number D1 of first-category image sets; each first-category image set corresponds to an ultrasound technology type; each first-category image set includes a plurality of the training cardiac ultrasound images; and each of the training cardiac ultrasound images corresponds to a first slice integrity level label; Step 92: Initialize the first set index to 1, initialize the first image number threshold to a preset third training number threshold, and initialize the first training state flag to a single set training flag; the first training state flag includes a single set training flag and a full set training flag; Step 93: Set the third training counter to 0; and use the first type of image set corresponding to the first set index as the corresponding current image set; Step 94: Select any one of the training cardiac ultrasound images from the current image set as a corresponding fourth training image; and perform truth value tensor setting based on the first slice integrity level label of the fourth training image to generate a corresponding fourth truth value vector; Step 95: Input the fourth training image into the second feature extraction module of the quality inspection model to perform feature extraction to generate the corresponding second feature map; input the second feature map into the section integrity classification module to perform section integrity level classification processing to obtain a corresponding fourth training classification vector; and substitute the fourth training classification vector and the fourth true value vector into a preset second regression loss function to calculate and obtain a corresponding third loss value; Step 96: Identify whether the third loss value satisfies a preset third loss value convergence range. When the third loss value satisfies the third loss value convergence range, increment the third training counter by 1, and identify whether the new third training counter exceeds the first image quantity threshold. If so, proceed to step 97; otherwise, proceed to step 94 to continue training. When the third loss value does not satisfy the third loss value convergence range, modulate the model parameters of the second feature extraction module and the section integrity classification module, and when the modulation is completed, proceed to step 95 to continue training. Step 97, identifying the first training state flag; if the first training state flag is a single set training flag, adding 1 to the first set index and going to step 98; if the first training state flag is a full set training flag, going to step 99; Step 98: Identify whether the first set index is greater than the first number D1. If the first set index is less than or equal to the first number D1, proceed to step 93 to continue training. If the first set index is greater than the first number D1, merge the first number D1 of first-category image sets into a complete image set as the new current image set, reset the first image number threshold to the number of training cardiac ultrasound images in the current image set, reset the third training counter to 0, set the first training status flag to the full set training flag, and proceed to step 94 to continue training. Step 99: Stop model training of the second feature extraction module and the section integrity classification module.

10. The method for classifying cardiac ultrasound images according to claim 9, wherein: The fourth truth value vector is recorded as the fourth truth value vector The fourth truth vector The fourth true value probability of the fourth quantity D4 is included 1≤k≤D4; each of the fourth truth value probabilities Corresponding to a section completeness level; the fourth truth vector There is only one fourth true value probability The probability value of the fourth truth value is 1, the rest are 0, and the probability value is 1 The corresponding section integrity level matches the corresponding first section integrity level label; The fourth training classification vector is recorded as the fourth training classification vector Y4; the fourth training classification vector Y4 includes the fourth training probability y of the fourth quantity D4 4,k Each of the fourth training probabilities y 4,k Corresponding to a section integrity level; The second regression loss function includes MSE Loss loss function, MAE Loss loss function and Pseudo-HuberLoss loss function.

11. The method for classifying cardiac ultrasound images according to claim 6, wherein: The classifying the image quality level and the slice integrity level of the first ultrasound image based on the preset quality detection model to generate a corresponding second classification vector specifically includes: The second feature extraction module of the quality detection model performs feature extraction processing on the input first ultrasound image to generate a corresponding second feature map and sends it to the image quality classification module and the section integrity classification module; The image quality classification module performs image quality classification processing on the second feature map to generate a corresponding first quality level classification vector and sends it to the second classification fusion module; the first quality level classification vector includes the third number D3 of the first quality level classification probabilities; The slice integrity classification module performs slice integrity level classification processing on the second feature map to generate a corresponding first integrity level classification vector and sends it to the second classification fusion module; the first slice classification vector includes the first integrity level classification probability of the fourth number D4; The second classification fusion module takes each first quality level classification probability of the first quality level classification vector as the corresponding first level probability, and the first level probabilities of the third quantity D3 obtained are used to form the corresponding first classification sub-vector; and takes each first integrity level classification probability of the first integrity level classification vector as the corresponding second level probability, and the second level probabilities of the fourth quantity D4 obtained are used to form the corresponding second classification sub-vector; and the first and second sub-vectors obtained are used to form the corresponding second classification vector output.

12. A device for implementing the method for classifying cardiac ultrasound images according to any one of claims 1 to 11, characterized in that: The device comprises: a receiving module, an M-type classification module, a quality detection module and a data classification module; The receiving module is configured to receive a cardiac ultrasound image as a corresponding first ultrasound image; The M-type classification module is used to classify the ultrasound technology type and the section type of the first ultrasound image based on a preset M-type classification model to generate a corresponding first classification vector; the number of ultrasound technology types in the M-type classification model is a first number D1, the number of section types in the M-type classification model is a second number D2, and the number M is the total number of model categories of the M-type classification model, D1 ≥ 1, D2 ≥ 1, M = D1 * D2 ≥ 1; the first classification vector includes the first classification probabilities of the number M, each of the first classification probabilities corresponding to a type of ultrasound technology type and a type of section type; The quality detection module is configured to classify the image quality level and the slice integrity level of the first ultrasound image based on a preset quality detection model to generate a corresponding second classification vector; the number of image quality levels of the quality detection model is a third number D3, and the number of slice integrity levels of the quality detection model is a fourth number D4; the second classification vector includes a first classification sub-vector and a second classification sub-vector; the first classification sub-vector includes first-level probabilities of the third number D3, each first-level probability corresponding to an image quality level; the second classification sub-vector includes second-level probabilities of the fourth number D4, each second-level probability corresponding to a slice integrity level; The data classification module is used to perform image data classification processing on the first ultrasound image according to the first and second classification vectors.

13. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1 to 11; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

Images