Echocardiogram image section type recognition method, electronic device and storage medium

Abstract

The application provides an echocardiogram section type recognition method, an electronic device and a storage medium. The recognition method comprises the following steps: obtaining an echocardiogram image to be recognized; preprocessing the echocardiogram image to be recognized to obtain a region of interest image; using a trained echocardiogram section type recognition model to recognize the region of interest image to obtain an output probability value of the echocardiogram image to be recognized belonging to each preset section type; and performing normalization processing on the output probability value of the echocardiogram image to be recognized belonging to each preset section type to obtain a normalized probability value of the echocardiogram image to be recognized belonging to each preset section type and other section types, so as to obtain a final recognition result of the section type of the echocardiogram image to be recognized. The application can solve the recognition problem of other section types without the need of specially preparing data of other types of sections for training of the echocardiogram section type recognition model.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for identifying the cross-sectional type of echocardiogram images, an electronic device, and a storage medium. Background Technology

[0002] Echocardiography is a non-invasive examination that uses ultrasound to examine the structure and function of the cardiovascular system. It is an essential examination for cardiovascular disease screening and diagnosis. Different structures of the heart are presented in corresponding cardiac sections, allowing for the evaluation of the size, shape, and function of these structures. Currently, the selection of echocardiographic sections relies heavily on the physician's clinical experience and requires a high level of technical skill. For inexperienced physicians or technicians, accurately locating the cardiac sections can be difficult.

[0003] With the development of image processing technology, the determination of standard sections in echocardiogram images based on image algorithms has seen some progress. For example, target detection-based standard section judgment methods can be used to detect targets in echocardiogram images, obtaining the structural type and corresponding confidence level of each target cardiac structure in the image to be detected. Based on this target information, the type of target cardiac section and whether it is a standard section can be determined. This method can improve the reliability of cardiac standard section detection results, but it requires labeling cardiac structures such as the left ventricle, left atrium, right ventricle, right atrium, spine, descending aorta, aortic arch, ribs, and spine, which consumes a lot of manpower and resources, limiting the widespread application of this method.

[0004] It should be noted that the information disclosed in the background section of this invention is intended only to enhance the understanding of the general background of this invention, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, electronic device, and storage medium for identifying the section type of echocardiograms, which can automatically identify the section type of echocardiograms to reduce the difficulty in the process of identifying the section type of echocardiograms.

[0006] To achieve the above objectives, the present invention provides a method for identifying the type of echocardiographic image section, the method comprising:

[0007] Acquire the echocardiogram image to be identified;

[0008] The echocardiogram to be identified is preprocessed to remove the background region and obtain the region of interest image.

[0009] The trained echocardiogram section type recognition model is used to identify the region of interest image to obtain the output probability value of the echocardiogram image to be identified belonging to each preset section type;

[0010] The output probability value of the echocardiogram to be identified belonging to each preset section type is normalized to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset section type and other section types, thereby obtaining the final identification result of the section type of the echocardiogram to be identified.

[0011] Optionally, the echocardiogram section type recognition model includes a feature extraction network and a classification network, wherein the feature extraction network is used to extract section features from the echocardiogram to be identified;

[0012] The classification network includes several fully connected layers and several softmax layers. Each fully connected layer is connected to a softmax layer. The number of fully connected layers and the number of softmax layers are equal to the number of categories of the preset section type. The fully connected layers are used to perform nonlinear mapping regression on the extracted section features. The softmax layers are used to obtain the output probability value of the echocardiogram image to be identified belonging to the corresponding preset section type based on the result of the nonlinear mapping regression of the fully connected layers connected to them.

[0013] Optionally, the preprocessing of the echocardiogram to be identified to remove background regions and obtain a region of interest image includes:

[0014] Obtain the region of interest mask corresponding to the echocardiogram image to be identified, wherein in the region of interest mask, the pixel value of the background region is 0, and the pixel value of the region of interest is 1;

[0015] The region of interest (ROI) image is obtained by multiplying the echocardiogram image to be identified with the region of interest mask.

[0016] Optionally, obtaining the mask of interest corresponding to the echocardiogram image to be identified includes:

[0017] The pixels of all frames in the echocardiogram video corresponding to the echocardiogram image to be identified are analyzed. The region whose pixel value remains unchanged along the video time axis is taken as the background region, and the region outside the background region is taken as the region of interest, so as to obtain the region of interest mask corresponding to the ultrasound video.

[0018] The region of interest mask corresponding to the echocardiogram video is used as the region of interest mask corresponding to the echocardiogram image to be identified.

[0019] Optionally, the preset section type includes a preset major section type and a preset minor section type corresponding to the preset major section type, wherein the minor section corresponding to the preset minor section type is the standard section of the major section corresponding to the preset major section type.

[0020] The final identification result of the section type of the echocardiogram to be identified includes the identification result of the major section type to which the echocardiogram to be identified belongs and the identification result of whether it is a sub-section type of the standard section of the major section type.

[0021] Optionally, the step of normalizing the output probability value of the echocardiogram to be identified belonging to each preset section type to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset section type and other section types, thereby obtaining the final identification result of the section type of the echocardiogram to be identified, includes:

[0022] The output probability values ​​of the echocardiogram to be identified belonging to each preset major section type are normalized to obtain the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types. Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types, the final identification result of the major section type of the echocardiogram to be identified is obtained, wherein the other major section types refer to major section types other than all preset major section types.

[0023] The output probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type are normalized to obtain the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types. Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types, the final identification result of the sub-section type of the echocardiogram to be identified is obtained, wherein the non-standard section is.

[0024] Optionally, the normalization processing of the output probability value of the echocardiogram to be identified belonging to each preset major section type to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset major section type and other major section types includes:

[0025] The output probability values ​​of the echocardiogram image to be identified belonging to each preset major section type are normalized according to the following formula to obtain the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset major section type:

[0026]

[0027]

[0028] In the formula, P i P is the output probability value of the echocardiogram to be identified belonging to the i-th preset major category section type. i "T" represents the normalized probability value of the echocardiogram image to be identified belonging to the i-th preset major category of section type. i The threshold for determining whether the echocardiogram to be identified belongs to the i-th preset major category of cross-section type is used for the echocardiogram cross-section type recognition model, where N is the number of preset major category cross-section types, and i is a positive integer;

[0029] Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type, the normalized probability values ​​of the echocardiogram to be identified belonging to other major section types are calculated.

[0030] Optionally, the step of obtaining the final identification result of the major section type of the echocardiogram to be identified based on the normalized probability values ​​of the major section type to which the echocardiogram to be identified belongs and other major section types includes:

[0031] Based on the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset major section type and other major section types, the major section type with the largest normalized probability value is taken as the final identification result of the major section type of the echocardiogram image to be identified.

[0032] Optionally, the normalization processing of the output probability value of the echocardiogram to be identified belonging to each preset sub-section type to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types includes:

[0033] The output probability value of the echocardiogram image to be identified belonging to each preset sub-category of section is normalized according to the following formula to obtain the normalized probability value of the echocardiogram image to be identified belonging to each preset sub-category of section:

[0034]

[0035]

[0036] In the formula, p i p″ is the output probability value of the echocardiogram to be identified belonging to the i-th preset sub-category section type. it is the normalized probability value of the echocardiogram to be identified belonging to the i-th preset sub-class section type. i The threshold for determining whether the echocardiogram to be identified belongs to the i-th preset sub-category of the echocardiogram is used by the echocardiogram section type recognition model, where N is the number of preset sub-category of the section type, and i is a positive integer;

[0037] Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type, the normalized probability values ​​of the echocardiogram to be identified belonging to other sub-section types are calculated.

[0038] Optionally, the step of obtaining the final identification result of the sub-section type of the echocardiogram to be identified based on the normalized probability values ​​of the sub-section type to which the echocardiogram to be identified belongs and other sub-section types includes:

[0039] Based on the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset sub-section type and other sub-section types, the sub-section type with the largest normalized probability value is taken as the candidate identification result of the sub-section type of the echocardiogram image to be identified.

[0040] Determine whether the major category of the candidate recognition result of the minor category of the cross section is consistent with the final recognition result of the major category of the ultrasound image to be recognized;

[0041] If so, the candidate recognition result of the sub-section type is taken as the final recognition result of the sub-section type of the echocardiogram to be recognized.

[0042] Optionally, if the major category of the candidate recognition result of the minor category of the cross-section is inconsistent with the final recognition result of the major category of the ultrasound image to be recognized, the recognition method further includes:

[0043] Based on the final identification result of the major category section type of the echocardiogram to be identified, the preset minor category section type corresponding to the final identification result of the major category section type is taken as the target minor category section type.

[0044] Determine whether the output probability value of the target sub-category section type is greater than or equal to the threshold value used by the echocardiogram section type recognition model to determine whether the ultrasound image to be identified belongs to the target sub-category section type;

[0045] If so, the target sub-category section type is taken as the final identification result of the sub-category section type corresponding to the echocardiogram to be identified;

[0046] If not, other sub-section types will be used as the final identification result of the sub-section type corresponding to the ultrasound image to be identified.

[0047] To achieve the above objectives, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the echocardiogram section type recognition method described above.

[0048] To achieve the above objectives, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the echocardiogram section type recognition method described above.

[0049] Compared with existing technologies, the echocardiogram slice type recognition method, electronic device, and storage medium provided by the present invention have the following advantages: The present invention first acquires an echocardiogram image to be identified; then preprocesses the echocardiogram image to be identified to remove the background region and obtain a region of interest image; then uses a trained echocardiogram slice type recognition model to identify the region of interest image to obtain the output probability value of the echocardiogram image to be identified belonging to each preset slice type; finally, normalizes the output probability value of the echocardiogram image to be identified belonging to each preset slice type to obtain the normalized probability value of the echocardiogram image to be identified belonging to each preset slice type and other slice types, thereby obtaining the final recognition result of the slice type of the echocardiogram image to be identified. Therefore, this invention can automatically identify the section type of echocardiogram images. This not only effectively improves algorithm accuracy, but also, because this invention directly uses a trained echocardiogram section type identification model to identify the echocardiogram image to be identified, it obtains the output probability value of the echocardiogram image belonging to each preset section type. By normalizing the output probability value of the echocardiogram image belonging to each preset section type, it obtains the normalized probability value of the echocardiogram image belonging to each preset section type and other section types, thereby obtaining the final identification result of the section type of the echocardiogram image to be identified. Therefore, compared to the existing method of using target detection standard section judgment to perform target detection on echocardiogram images to obtain the structural type and corresponding confidence level of each target cardiac structure in the target cardiac image, and then determining the type of the target cardiac section and whether it is a standard section based on this target information, this invention does not require a large amount of organ structure (e.g., cardiac structure) annotation process, making it easier to identify the section type of echocardiogram images. Furthermore, the echocardiogram section type recognition method provided by this invention has strong algorithm versatility and implements an end-to-end algorithm flow, which can better assist doctors in identifying section types and reduce the difficulty in the echocardiogram section type recognition process. In addition, by normalizing the output probability values ​​of the echocardiogram to be identified belonging to each preset section type output by the echocardiogram section type recognition model, this invention can obtain normalized probability values ​​of the echocardiogram to be identified belonging to other section types. Therefore, it solves the problem of identifying other section types without needing to specifically prepare data for other section types for training the echocardiogram section type recognition model. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a method for identifying echocardiographic image section types according to an embodiment of the present invention.

[0051] Figure 2a This is a schematic diagram of the echocardiogram to be identified in a specific example of the present invention;

[0052] Figure 2b for Figure 2a A schematic diagram of the region of interest mask corresponding to the echocardiogram to be identified;

[0053] Figure 2c for Figure 2a A schematic diagram of the region of interest corresponding to the echocardiogram image to be identified;

[0054] Figure 3 This is a schematic diagram of the structure of a densely connected block in a specific example;

[0055] Figure 4 This is a schematic diagram of the structure of an echocardiogram section type recognition model in a specific example of the present invention;

[0056] Figure 5 This is a schematic diagram of the bottleneck layer in a specific example of the present invention;

[0057] Figure 6 This is a schematic diagram of the transition block in a specific example of the present invention;

[0058] Figure 7 A schematic diagram illustrating the classification principle provided as a specific example of the present invention;

[0059] Figure 8 This is a schematic diagram showing the result of section type identification using the echocardiogram section type identification method provided by the present invention;

[0060] Figure 9 This is a block diagram of an electronic device according to one embodiment of the present invention;

[0061] The accompanying figure is labeled as follows:

[0062] Processor-101; Communication interface-102; Memory-103; Communication bus-104. Detailed Implementation

[0063] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the method for identifying echocardiographic slice types, the electronic device, and the storage medium proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the embodiments of this invention. Please refer to the drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for illustrative purposes and to enable those skilled in the art to understand and read them, and are not intended to limit the implementation conditions of this invention. Any modifications to the structure, changes in proportions, or adjustments to the size, provided they produce the same or similar effects and achieve the same objectives as this invention, should still fall within the scope of the technical content disclosed in this invention.

[0064] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0065] Furthermore, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0066] The core idea of ​​this invention is to provide a method, electronic device, and storage medium for identifying the slice type of echocardiogram images, which can automatically identify the slice type of echocardiogram images, thereby reducing the difficulty in the slice type identification process. It should be noted that slice type identification in this invention refers to identifying the type of cardiac slice corresponding to the echocardiogram image acquisition. The term "trained echocardiogram slice type identification model" refers to the model whose parameters are continuously updated by training a pre-built echocardiogram slice type identification model, including a feature extraction network and a classification network, using training samples until a preset training termination condition is met. In other words, the model parameters of the trained echocardiogram slice type identification model are the model parameters at the end of training.

[0067] Furthermore, it should be noted that the echocardiogram section type recognition method of the present invention can be applied to the electronic device of the present invention, wherein the electronic device can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a mobile phone, a tablet computer, or other hardware device with various operating systems.

[0068] To achieve the above-mentioned goals, this invention provides a method for identifying the type of echocardiographic image section. Please refer to [the relevant documentation]. Figure 1 The diagram illustrates a flowchart of a method for identifying echocardiographic image section types according to an embodiment of the present invention. Figure 1 As shown, the method for identifying the section type of echocardiogram includes the following steps:

[0069] Step S100: Obtain the echocardiogram image to be identified.

[0070] Step S200: Preprocess the echocardiogram to be identified to remove the background region in the echocardiogram to be identified and obtain the region of interest image.

[0071] Step S300: The trained echocardiogram section type recognition model is used to identify the region of interest image to obtain the output probability value of the echocardiogram to be identified belonging to each preset section type.

[0072] Step S400: Normalize the output probability value of the echocardiogram to be identified belonging to each preset section type to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset section type and other section types, thereby obtaining the final identification result of the section type of the echocardiogram to be identified.

[0073] Therefore, this invention first preprocesses the acquired echocardiogram image to remove background regions containing irrelevant information, obtaining a region of interest (ROI) image. Then, a pre-trained echocardiogram section type recognition model is used to identify the ROI image. Since the ROI image does not contain irrelevant information, this effectively improves the efficiency and accuracy of identifying the section type of the echocardiogram image. Furthermore, the recognition method provided by this invention normalizes the output probability values ​​of the echocardiogram image belonging to each preset section type, obtained from the echocardiogram section type recognition model, to obtain normalized probability values ​​for each preset section type and other section types. This yields the final identification result of the section type of the echocardiogram image, thus solving the problem of identifying other section types without requiring specially prepared data for training the echocardiogram section type recognition model. Furthermore, since the identification method provided by this invention can automatically identify the section type of echocardiogram images, it can not only effectively improve the accuracy of the algorithm, but also directly use a trained echocardiogram section type identification model to identify the echocardiogram image to be identified, to obtain the output probability value of the echocardiogram image to be identified belonging to each preset section type. By normalizing the output probability value of the echocardiogram image to be identified belonging to each preset section type, the normalized probability value of the echocardiogram image to be identified belonging to each preset section type and other section types is obtained, thereby obtaining the final identification result of the section type of the echocardiogram image to be identified. Therefore, compared with the existing method of target detection standard section judgment to perform target detection on echocardiogram images to obtain the structural type and corresponding confidence of each target heart structure in the heart image to be detected, and then determine the type of the target heart section and whether it is a standard section based on these target information, the identification method provided by this invention does not require a large number of organ structure (e.g., heart structure) annotation processes, making it easier to realize the identification of echocardiogram image section type. Furthermore, the method for identifying the section type of echocardiogram provided by this invention has strong algorithm versatility and implements an end-to-end algorithm flow, which can better assist doctors in identifying the section type and reduce the difficulty in the process of identifying the section type of echocardiogram.

[0074] Specifically, the echocardiogram image to be identified is extracted from the acquired echocardiogram video (e.g., echocardiogram video), and the video resolution can be set according to specific circumstances, such as 600×800.

[0075] In one exemplary embodiment, the preprocessing of the echocardiogram to be identified to remove background regions and obtain a region of interest image includes:

[0076] Obtain the region of interest mask corresponding to the echocardiogram image to be identified, wherein in the region of interest mask, the pixel value of the background region is 0, and the pixel value of the region of interest is 1;

[0077] The region of interest (ROI) image is obtained by multiplying the echocardiogram image to be identified with the region of interest mask.

[0078] Please refer to Figures 2a to 2c ,in, Figure 2a This is a schematic diagram of the echocardiogram to be identified in a specific example of the present invention; Figure 2b for Figure 2a A schematic diagram of the region of interest mask corresponding to the echocardiogram to be identified; Figure 2c for Figure 2a This is a schematic diagram of the region of interest corresponding to the echocardiogram image to be identified. Figures 2a to 2c As shown, by performing a multiplication operation between the echocardiogram image to be identified and the region of interest mask (multiplying the pixel value of each pixel in the echocardiogram image to be identified with the pixel value of the corresponding pixel in the region of interest mask), the background region in the echocardiogram image to be identified can be removed (i.e., the background region in the echocardiogram image to be identified is set to black), and only the region in the fan-shaped area (i.e., the region of interest) is retained, thereby obtaining the region of interest image corresponding to the echocardiogram image to be identified.

[0079] In one exemplary embodiment, acquiring the mask of interest corresponding to the echocardiogram to be identified includes:

[0080] The pixels in all frames of the echocardiogram video corresponding to the echocardiogram image to be identified are analyzed. The region whose pixel value remains unchanged along the video time axis is taken as the background region, and the region outside the background region is taken as the region of interest, so as to obtain the region of interest mask corresponding to the echocardiogram video.

[0081] The region of interest mask corresponding to the echocardiogram video is used as the region of interest mask corresponding to the echocardiogram image to be identified.

[0082] Since the pixel values ​​of the background region are consistent across all frames of the same echocardiogram video, by analyzing the pixels in all frames of the echocardiogram video corresponding to the image to be identified, the region whose pixel value remains unchanged along the video time axis (i.e., the region whose pixel value remains unchanged across all frames of the echocardiogram image) can be identified. This region is the background region, and the region outside the background region (i.e., the region within the fan-shaped area) is the region of interest. Based on the determined background region and region of interest, the region of interest mask corresponding to the ultrasound video can be obtained, and the region of interest mask corresponding to the ultrasound video is the region of interest mask corresponding to the echocardiogram image to be identified. Therefore, this invention, by employing a temporal depth method to obtain the region of interest mask corresponding to the echocardiogram image to be identified, can effectively reduce the cost of obtaining the region of interest mask corresponding to the echocardiogram image to be identified. Of course, as those skilled in the art will understand, in other embodiments, existing image segmentation methods (such as threshold segmentation, region growing, and deep learning-based image segmentation) can also be used to segment the echocardiogram image to be identified in order to obtain a mask of the region of interest corresponding to the echocardiogram image to be identified.

[0083] In one exemplary embodiment, the echocardiogram section type recognition model includes a feature extraction network and a classification network. The feature extraction network is used to extract section features from the ultrasound image to be identified, and the classification network is used to perform nonlinear mapping regression on the extracted section features to obtain the output probability value of the echocardiogram to be identified belonging to each preset section type.

[0084] Specifically, the feature extraction network can employ the DenseNet network. Since the DenseNet network is a densely connected convolutional neural network, with each layer's input derived from the outputs of all preceding layers, this neural network structure enhances feature propagation and utilizes features more effectively. Furthermore, the DenseNet neural network model exhibits good anti-overfitting performance, making it particularly suitable for applications with relatively scarce training data. Therefore, using the DenseNet network as the feature extraction network in this invention enables the reuse of cross-sectional features along the channel dimension, maximizing information exchange between layers. This not only alleviates the gradient vanishing phenomenon but also allows for superior performance with fewer parameters and computational cost, thereby improving the accuracy of cross-sectional type recognition. Specifically, the DenseNet network consists of multiple densely connected blocks connected by transition blocks; that is, any two adjacent densely connected blocks are connected by a transition block, and the number of convolutional output channels within each densely connected block is consistent, facilitating the superposition of feature information from each layer.

[0085] One layer in a densely connected block is called a bottleneck layer. Dense connections in DenseNet connect each layer in a densely connected block to all subsequent layers, enabling feature reuse. Please refer to [link / reference]. Figure 3 It schematically illustrates the structure of a densely connected block in a specific example. For example... Figure 3 As shown, suppose a densely connected block has L bottleneck layers, X0 is the input of the densely connected block (defined as the output of layer 0), and layer L combines the outputs of all previous layers X0, ..., X... L-1 As input, the input of layer L satisfies the following relationship with the outputs of all previous layers:

[0086] X L =H L ([X0,X1,…X L-1 ])

[0087] Where, [X0,X1,…X L-1 [] indicates that the outputs from layers 0 to (L-1) are combined and concatenated to serve as the input to the bottleneck layer of layer L. H L This represents all operations of the Lth bottleneck layer. Specifically, each bottleneck layer includes multiple operations: batch normalization (BN), ReLU activation function, and 3×3 convolution.

[0088] Please continue to refer to this. Figure 4 The diagram illustrates the structure of an echocardiogram section type recognition model in a specific example of the present invention. Figure 4 As shown in this example, the feature extraction network includes a first convolutional layer, a first dense connection block, a first transition block, a second dense connection block, a second transition block, a third dense connection block, a third transition block, a fourth dense connection block, and a first pooling layer (preferably a max pooling layer) connected in sequence. Specifically, the first convolutional layer extracts cross-sectional features from the input ultrasound image to be identified; the first dense connection block extracts cross-sectional features from the output of the first convolutional layer; the first transition block compresses the output of the first dense connection block; the second dense connection block extracts cross-sectional features from the output of the first transition block; the third dense connection block extracts cross-sectional features from the output of the second transition block; the fourth dense connection block extracts cross-sectional features from the output of the third transition block; and the first pooling layer performs pooling (preferably max pooling) on ​​the output of the fourth dense connection block.

[0089] It should be noted that, as those skilled in the art will understand, the first dense connection block, the second dense connection block, the third dense connection block, and the fourth dense connection block all include multiple bottleneck layers, and the number of bottleneck layers in the first dense connection block, the second dense connection block, the third dense connection block, and the fourth dense connection block can be the same or different. The specific number can be set according to actual needs, and the present invention does not limit this. For example, the first dense connection block may have 6 bottleneck layers, the second dense connection block may have 12 bottleneck layers, the third dense connection block may have 24 bottleneck layers, and the fourth dense connection block may have 16 bottleneck layers.

[0090] Please continue to refer to this. Figure 5 The diagram illustrates the structure of the bottleneck layer in a specific example of the present invention. Figure 5 As shown, the bottleneck layer comprises a first batch normalization layer A, a first activation layer A, a second convolutional layer A, a first batch normalization layer B, a first activation layer B, and a second convolutional layer B connected in sequence. The kernel size of the second convolutional layer A is 1×1, and the kernel size of the second convolutional layer B is 3×3. Therefore, by adding a 1×1 convolution before the 3×3 convolution in the bottleneck layer, this invention can reduce the number of feature maps and lower the dimensionality of each feature map, thereby reducing computational cost and fusing features from various channels. Furthermore, since the bottleneck layer performs batch normalization (BN) and ReLU activation operations before both the 1×1 and 3×3 convolution operations, training speed and convergence efficiency can be further improved.

[0091] Please continue to refer to this. Figure 6 The diagram illustrates the structure of a transition block in a specific example of the present invention. Figure 6 As shown, the first transition block, the second transition block, and the third transition block each include a second batch normalization layer, a second activation layer, a third convolutional layer, and a second pooling layer (preferably an average pooling layer) connected in sequence. The kernel size of the third convolutional layer is 1×1. Thus, dimensionality reduction is achieved through the convolution operation of the third convolutional layer, and the average pooling operation of the fourth pooling layer solves the problem of excessive channel count, preventing model complexity caused by too many densely connected blocks. Furthermore, since each transition block performs batch normalization (BN) and ReLU activation operations before the 1×1 convolution operation, the number of parameters can be further reduced.

[0092] In one exemplary embodiment, the classification network includes several fully connected layers and several softmax layers, each of the fully connected layers being connected to a softmax layer. The number of fully connected layers and the number of softmax layers are equal to the number of categories of a preset section type. The fully connected layers are used to perform nonlinear mapping regression on the extracted section features, and the softmax layers are used to obtain the output probability value of the echocardiogram image to be identified belonging to the corresponding preset section type based on the result of the nonlinear mapping regression of the fully connected layers connected to them.

[0093] It should be noted that, as those skilled in the art will understand, the number of fully connected layers and softmax layers included in the classification network can be set according to specific circumstances, as long as it is equal to the number of categories of the preset facet type. The present invention does not limit this. For example, when the number of categories of the preset facet type is 10, the classification network includes 10 fully connected layers and 10 softmax layers.

[0094] In one exemplary embodiment, the preset section type includes a preset major section type and a preset minor section type corresponding to the preset major section type, wherein the minor section corresponding to the preset minor section type is the standard section of the major section corresponding to the preset major section type.

[0095] The final identification result of the section type includes the identification result of the major section type to which the echocardiogram to be identified belongs and the identification result of whether it is a sub-section type of the standard section of the major section type.

[0096] Therefore, the echocardiogram section type identification method provided by this invention can identify both the major section type corresponding to the echocardiogram and whether the minor section corresponding to the echocardiogram is a standard section of the major section type. Please refer to... Figure 7 The diagram illustrates the classification principle provided by a specific example of the present invention. Specifically, it uses the plax (parasternal long axis) section, psax_mv (parasternal short axis (mitral valve level)) section, a2c (apical two-chamber) section, a3c (apical three-chamber) section, and a4c (apical four-chamber) section as examples of five preset major section types, such as... Figure 7As shown, the classification network is designed with 10 binary classification tasks (i.e., the classification network includes 10 fully connected layers and 10 softmax layers), namely: act_plax: whether it is a plax aspect (major category aspect type); act_psax_mv: whether it is a psax_mv aspect (major category aspect type); act_a2c: whether it is an a2c aspect (major category aspect type); act_a3c: whether it is an a3c aspect (major category aspect type); act_a4c: whether it is an a... 4c aspect (major category aspect type); act_std_plax: whether it is a plax standard aspect (minor category aspect type); act_std_psax_mv: whether it is a psax_mv standard aspect (minor category aspect type); act_std_a2c: whether it is an a2c standard aspect (minor category aspect type); act_std_a3c: whether it is an a3c standard aspect (minor category aspect type); act_std_a4c: whether it is an a4c standard aspect (minor category aspect type). Therefore, this setting allows for the direct calculation of the probability that an ultrasound image does not belong to one of the five preset cross-sectional types without the need for an additional "other cross-section" (other cross-sectional types, different from the plax, psax_mv, a2c, a3c, and a4c cross-sections). This eliminates the need for an additional "other cross-sectional" dataset during the training of the echocardiogram cross-sectional type recognition model, thereby effectively reducing the training cost and manpower costs during data collection.

[0097] Therefore, based on the nonlinear mapping regression results of the corresponding fully connected layers, each softmax layer can calculate whether the ultrasound image to be identified is a plax section (preset major section type), a psax_mv section (preset major section type), an a2c section (preset major section type), an a3c section (preset major section type), an a4c section (preset major section type), a plax standard section (preset minor section type), a psax_mv standard section (preset minor section type), an a2c standard section (preset minor section type), an a3c standard section (preset minor section type), or an a4c standard section. The output probability value of each of the 10 preset section types (preset sub-section types) is obtained. After obtaining the above output probability values, the recognition result of the major section type corresponding to the ultrasound image to be recognized (which one it is: plax section, psax_mv section, a2c section, a3c section, a4c section, or other major section types) can be obtained by normalizing the output probability values ​​of each sub-section type. The recognition result of the sub-section type corresponding to the ultrasound image to be recognized (i.e., whether it is the standard section of the major section type of that type) can be obtained by normalizing the output probability values ​​of each sub-section type. It should be noted that, as those skilled in the art will understand, when the major section type corresponding to the ultrasound image to be recognized is identified as another major section type, it is not necessary to obtain the recognition result of the sub-section type corresponding to the ultrasound image to be recognized.

[0098] In one exemplary embodiment, the normalization processing of the output probability value of the echocardiogram to be identified belonging to each preset section type to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset section type and other section types, thereby obtaining the final identification result of the section type of the echocardiogram to be identified, includes:

[0099] The output probability values ​​of the echocardiogram to be identified belonging to each preset major section type are normalized to obtain the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types. Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types, the final identification result of the major section type of the echocardiogram to be identified is obtained, wherein the other major section types refer to major section types other than all preset major section types.

[0100] The output probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type are normalized to obtain the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types. Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types, the final identification result of the sub-section type of the echocardiogram to be identified is obtained, wherein the non-standard section is.

[0101] Therefore, by normalizing the output probability values ​​of the echocardiogram to be identified belonging to each preset major section type, the sum of the probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types can be normalized to 1. For example, by normalizing the output probability values ​​of the echocardiogram to be identified belonging to the plax section, psax_mv section, a2c section, a3c section, and a4c section, the sum of the normalized probability values ​​of the echocardiogram to be identified belonging to the plax section, psax_mv section, a2c section, a3c section, a4c section, and other major section types can be made to 1. That is, the normalized probability value of other major section types = 1 - the sum of the normalized probability values ​​of all preset major section types. Similarly, by normalizing the output probability value of each preset sub-category of section type, the sum of the probability values ​​of all preset sub-category sections and other sub-category sections can be normalized to 1. For example, by normalizing the output probability values ​​of the echocardiogram to be identified belonging to the plax standard section, psax_mv standard section, a2c standard section, a3c standard section, and a4c standard section, the sum of the normalized probability values ​​of the echocardiogram to be identified belonging to the plax standard section, psax_mv standard section, a2c standard section, a3c standard section, a4c standard section, and other sub-category sections (non-standard sections) can be made to 1.

[0102] Further, the normalization processing of the output probability values ​​of the echocardiogram to be identified belonging to each preset major section type to obtain the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types includes:

[0103] The output probability values ​​of the echocardiogram image to be identified belonging to each preset major section type are normalized according to the following formula to obtain the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset major section type:

[0104]

[0105]

[0106] In the formula, P i P is the output probability value of the echocardiogram to be identified belonging to the i-th preset major category section type. i "T" represents the normalized probability value of the echocardiogram image to be identified belonging to the i-th preset major category of section type. i The threshold for determining whether the echocardiogram to be identified belongs to the i-th preset major category of cross-section type is used for the echocardiogram cross-section type recognition model, where N is the number of preset major category cross-section types, and i is a positive integer;

[0107] Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type, the normalized probability values ​​of the echocardiogram to be identified belonging to other major section types are calculated.

[0108] Therefore, using the above formula, the sum of the probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types can be normalized to 1. For example, by normalizing the output probability values ​​of the echocardiogram to be identified belonging to the plax section, psax_mv section, a2c section, a3c section, and a4c section respectively, the sum of the normalized probability values ​​of the echocardiogram to be identified belonging to the plax section, psax_mv section, a2c section, a3c section, a4c section, and other major section types can be made to 1. That is, the normalized probability value of other major section types = 1 - the sum of the normalized probability values ​​of all preset major section types. It should be noted that, as those skilled in the art will understand, T i Corresponding to the maximum value of Youden's J statistic (i.e., tpr-fpr), each preset major category of section type corresponds to a threshold T. i .

[0109] In one exemplary embodiment, obtaining the final identification result of the major section type of the echocardiogram to be identified based on the normalized probability values ​​of the major section type to which the echocardiogram to be identified belongs and other major section types includes:

[0110] Based on the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset major section type and other major section types, the major section type with the largest normalized probability value is taken as the final identification result of the major section type of the echocardiogram image to be identified.

[0111] Therefore, by comparing the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type with the normalized probability values ​​of other major section types, the major section type with the highest normalized probability value can be determined. This major section type is the final identification result of the major section type corresponding to the echocardiogram to be identified. For example, if by comparison, it is found that among the six major section types—plax, psax_mv, a2c, a3c, a4c, and other major section types—the normalized probability value of the a2c section is the highest, then the final identification result of the major section type corresponding to the echocardiogram to be identified is the a2c section.

[0112] In one exemplary embodiment, the normalization processing of the output probability value of the echocardiogram to be identified belonging to each preset sub-section type to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types includes:

[0113] The output probability value of the echocardiogram image to be identified belonging to each preset sub-category of section is normalized according to the following formula to obtain the normalized probability value of the echocardiogram image to be identified belonging to each preset sub-category of section:

[0114]

[0115]

[0116] In the formula, p i p″ is the output probability value of the echocardiogram to be identified belonging to the i-th preset sub-category section type. i t is the normalized probability value of the echocardiogram to be identified belonging to the i-th preset sub-class section type. i The threshold for determining whether the echocardiogram to be identified belongs to the i-th preset sub-category of the echocardiogram is used by the echocardiogram section type recognition model, where N is the number of preset sub-category of the section type, and i is a positive integer;

[0117] Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type, the normalized probability values ​​of the echocardiogram to be identified belonging to other sub-section types are calculated.

[0118] Therefore, using the above formula, the sum of the probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types can be normalized to 1. For example, by normalizing the output probability values ​​of the echocardiogram to be identified belonging to the plax standard section, psax_mv standard section, a2c standard section, a3c standard section, and a4c standard section respectively, the sum of the normalized probability values ​​of the echocardiogram to be identified belonging to the plax standard section, psax_mv standard section, a2c standard section, a3c standard section, a4c standard section, and other sub-section types (non-standard sections) can be made to 1. That is, the normalized probability value of other sub-section types = 1 - the sum of the normalized probability values ​​of all preset sub-section types. It should be noted that, as those skilled in the art will understand, t i Corresponding to the maximum value of Youden's J statistic (i.e., tpr-fpr), each preset subclass aspect type corresponds to a threshold t. i .

[0119] In one exemplary embodiment, obtaining the final identification result of the sub-section type of the echocardiogram to be identified based on the normalized probability values ​​of the sub-section type to which the echocardiogram to be identified belongs and other sub-section types includes:

[0120] Based on the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset sub-section type and other sub-section types, the sub-section type with the largest normalized probability value is taken as the candidate identification result of the sub-section type of the echocardiogram image to be identified.

[0121] Determine whether the major category of the candidate recognition result of the minor category of the cross section is consistent with the final recognition result of the major category of the ultrasound image to be recognized;

[0122] If so, the candidate recognition result of the sub-section type is taken as the final recognition result of the sub-section type of the echocardiogram to be recognized.

[0123] Therefore, by comparing the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset sub-section type with the normalized probability values ​​of other sub-section types, the sub-section type with the largest normalized probability value can be determined. This sub-section type is the candidate identification result of the sub-section type of the ultrasound image to be identified. Then, it is determined whether the major section type corresponding to the candidate identification result of the sub-section type is consistent with the identified major section type. If they are consistent, the candidate identification result of the sub-section type is directly used as the final identification result of the sub-section type corresponding to the ultrasound image to be identified. For example, if the identification result of the major section type corresponding to the ultrasound image to be identified is section a2c, by comparison, it is found that among the six minor section types—plax standard section, psax_mv standard section, a2c standard section, a3c standard section, a4c standard section, and other minor section types—the normalized probability value of the a2c standard section is the highest. Since the major section type corresponding to the a2c standard section is section a2c, which is consistent with the identification result of the major section type corresponding to the ultrasound image to be identified, it indicates that the minor section type corresponding to the ultrasound image to be identified is correct. The section type is the a2c standard section. If the normalized probability value of other sub-section types is found to be the largest, then the identification result of the major section type corresponding to the other sub-section type is assumed to be consistent with the identification result of the major section type corresponding to the ultrasound image to be identified, and thus the sub-section type corresponding to the ultrasound image to be identified is determined to be the a2c non-standard section. If the normalized probability value of one of the standard sections, such as the plax standard section, psax_mv standard section, a3c standard section, and a4c standard section, is found to be the largest, then the sub-section type is determined according to the method described below.

[0124] In one exemplary embodiment, if the major category of the candidate identification result of the minor category of the section type is inconsistent with the final identification result of the major category of the ultrasound image to be identified, the identification method further includes:

[0125] Based on the final recognition result of the major category section type of the echocardiogram to be identified, the preset minor category section type corresponding to the recognition result of the major category section type is taken as the target minor category section type.

[0126] Determine whether the output probability value of the target sub-category section type is greater than or equal to the threshold value used by the echocardiogram section type recognition model to determine whether the ultrasound image to be identified belongs to the target sub-category section type;

[0127] If so, the target sub-category section type is taken as the final identification result of the sub-category section type corresponding to the echocardiogram to be identified;

[0128] If not, other sub-section types will be used as the final identification result of the sub-section type corresponding to the ultrasound image to be identified.

[0129] For example, if the identification result of the major section type corresponding to the ultrasound image to be identified is the a2c section, then the a2c standard section is taken as the target minor section type, and the output probability value of the a2c standard section is compared with the threshold for the echocardiogram section type identification model to determine that the ultrasound image to be identified is the a2c standard section. If the output probability value of the a2c standard section is greater than or equal to the threshold for the echocardiogram section type identification model to determine that the ultrasound image to be identified is the a2c standard section, then the a2c standard section is taken as the identification result of the minor section type corresponding to the ultrasound image to be identified; if the output probability value of the a2c standard section is less than the threshold for the echocardiogram section type identification model to output the a2c standard section, then the a2c non-standard section is taken as the identification result of the minor section type corresponding to the ultrasound image to be identified.

[0130] In one exemplary embodiment, the echocardiogram section type identification method provided by the present invention further includes:

[0131] The final identification result of the cross-section type of the echocardiogram to be identified is plotted at an appropriate position on the echocardiogram to be identified.

[0132] Therefore, by displaying the final identification result of the section type of the echocardiogram to be identified at an appropriate position on the echocardiogram, it is easier for the echocardiogram acquisition personnel to judge and adjust the section scanning position. Please refer to... Figure 8 The diagram illustrates the results of section type recognition using the echocardiogram section type recognition method provided by this invention. In the diagram, `label` represents the true label, `pred` represents the prediction result of the echocardiogram section type recognition model, `prob` represents the normalized probability value, and `threshold` is the threshold at which the echocardiogram section type recognition model outputs a standard plax section. Figure 8 As shown, the actual section type of the echocardiogram in this example is a plax section and a standard plax section. The recognition result of the echocardiogram section type recognition method provided by the present invention is also a plax section (normalized probability value of 1) and a standard plax section (normalized probability value of 0.9). It can be seen that the echocardiogram section type recognition method provided by the present invention can accurately identify the section type corresponding to the ultrasound image and accurately determine whether it is a standard section of that type.

[0133] The training process of the echocardiogram section type recognition model in this invention will be described below.

[0134] Specifically, the echocardiogram section type recognition model is trained through the following process:

[0135] Obtain training samples, which include echocardiogram images and category labels corresponding to the echocardiogram images. The category labels include major section type category labels and minor section type category labels.

[0136] Set the initial values ​​for the model parameters of the echocardiogram section type recognition model;

[0137] Based on the training samples and the initial values ​​of the model parameters, the pre-built echocardiogram section type recognition model is trained until the preset training termination condition is met.

[0138] To facilitate training, the training data in this invention comes from standard and non-standard ultrasound videos (e.g., echocardiogram videos) of different types of cross-sections (major cross-section types). Image frames from the acquired echocardiogram videos and their corresponding category labels are used as training samples for model training. Since the ultrasound images between consecutive frames of the same ultrasound video are relatively similar, to increase the diversity of training data and the generalization ability of the model, each extracted echocardiogram image is first preprocessed to obtain a region of interest (ROI) image corresponding to that echocardiogram image. Then, the obtained ROI images undergo random rigid transformations: for example, adjusting image contrast through gamma transformation, horizontal flipping, vertical flipping, adding noise, random cropping, rotation, scaling, etc., thereby improving the diversity of training data. The preprocessed training samples (each training sample includes an echocardiogram image and its corresponding category label) are divided into training, validation, and test sets according to a certain ratio, for example, a 6:1:3 ratio. It should be noted that, as those skilled in the art will understand, the major section type acquired during echocardiographic video acquisition can be used as the category label for the major section type of each frame of ultrasound image in the ultrasound video, and the minor section type acquired during ultrasound video acquisition can be used as the category label for the minor section type of each frame of ultrasound image in the ultrasound video (whether it is a standard section of the major section type).

[0139] As those skilled in the art will understand, model parameters include two categories: feature parameters and hyperparameters. Feature parameters are used to learn cross-sectional features and are learned iteratively by the model. Hyperparameters are parameters that are set manually. By setting appropriate hyperparameters, the network can be trained well. Hyperparameters may include the learning rate, the number of hidden layers, the kernel size, the number of training iterations, and the batch size for each iteration. The learning rate can be considered as the stride. As an example, this invention can set the learning rate to 0.001, the number of training epochs to 50, and the batch size for each iteration to 16.

[0140] Since the training process of a model is actually the process of minimizing the loss function, and finding the derivative can quickly and easily achieve this goal, this method of finding the derivative is called gradient descent. Therefore, this invention can preferentially use gradient descent to update the model parameters until it converges to the error requirement (the loss function value no longer changes) or reaches the set EarlyStopping condition, and then save the model parameters at this time.

[0141] Since the loss function is the objective function used to optimize the model, minimizing this loss function helps the model learn better. The loss function chosen in this invention is the Focal Loss function. The Focal Loss function is a new loss function modified from the standard cross-entropy loss function. It reduces the loss weight of easily classified samples, allowing the model to focus more on difficult-to-classify samples during training. The formula for the Focal Loss function is shown below:

[0142] FL=-α t (1-p t ) γ log p t

[0143] In the formula, α t p represents the weight of a certain category. t For the output probability value of a certain category, the weight of the category can be increased for a small number of samples, (1-p) t ) γ p is the adjustment coefficient. t The larger the value, the smaller the adjustment coefficient, so the model will ignore this sample that has already performed well.

[0144] In one exemplary implementation, the Adam optimization method is used to train the echocardiogram image recognition model. Adam can quickly achieve excellent results and has a significant advantage over other types of stochastic optimization algorithms.

[0145] Based on the same inventive concept, the present invention also provides an electronic device, please refer to... Figure 9A block diagram illustrating an embodiment of the electronic device provided by the present invention is shown. Figure 9 As shown, the electronic device includes a processor 101 and a memory 103. The memory 103 stores a computer program. When the computer program is executed by the processor 101, it implements the echocardiogram section type recognition method described above. Since the electronic device provided by this invention and the echocardiogram section type recognition method provided by this invention belong to the same inventive concept, the electronic device provided by this invention has all the advantages of the echocardiogram section type recognition method provided by this invention. Therefore, the beneficial effects of the electronic device provided by this invention will not be elaborated further.

[0146] like Figure 9 As shown, the electronic device also includes a communication interface 102 and a communication bus 104, wherein the processor 101, the communication interface 102, and the memory 103 communicate with each other via the communication bus 104. The communication bus 104 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 104 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 102 is used for communication between the aforementioned electronic device and other devices.

[0147] The processor 101 referred to in this invention can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 101 is the control center of the electronic device, connecting various parts of the electronic device through various interfaces and lines.

[0148] The memory 103 can be used to store the computer program. The processor 101 implements various functions of the electronic device by running or executing the computer program stored in the memory 103 and calling the data stored in the memory 103.

[0149] The memory 103 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0150] This invention also provides a readable storage medium storing a computer program that, when executed by a processor, can implement the echocardiographic image section type recognition method described above. Since the readable storage medium provided by this invention and the echocardiographic image section type recognition method provided by this invention belong to the same inventive concept, the readable storage medium provided by this invention possesses all the advantages of the echocardiographic image section type recognition method provided by this invention. Therefore, the beneficial effects of the readable storage medium provided by this invention will not be elaborated further.

[0151] The readable storage medium of embodiments of the present invention can be any combination of one or more computer-readable media. The readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer hard disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0152] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0153] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0154] In summary, compared with the prior art, the echocardiogram slice type recognition method, electronic device, and storage medium provided by the present invention have the following advantages: The present invention first acquires an echocardiogram image to be identified; then preprocesses the echocardiogram image to be identified to remove the background region and obtain a region of interest image; then uses a trained echocardiogram slice type recognition model to identify the region of interest image to obtain the output probability value of the echocardiogram image to be identified belonging to each preset slice type; finally, normalizes the output probability value of the echocardiogram image to be identified belonging to each preset slice type to obtain the normalized probability value of the echocardiogram image to be identified belonging to each preset slice type and other slice types, thereby obtaining the final recognition result of the slice type of the echocardiogram image to be identified. Therefore, this invention can automatically identify the section type of echocardiogram images. This not only effectively improves algorithm accuracy, but also, because this invention directly uses a trained echocardiogram section type identification model to identify the echocardiogram image to be identified, it obtains the output probability value of the echocardiogram image belonging to each preset section type. By normalizing the output probability value of the echocardiogram image belonging to each preset section type, it obtains the normalized probability value of the echocardiogram image belonging to each preset section type and other section types, thereby obtaining the final identification result of the section type of the echocardiogram image to be identified. Therefore, compared to the existing method of using target detection standard section judgment to perform target detection on echocardiogram images to obtain the structural type and corresponding confidence level of each target cardiac structure in the target cardiac image, and then determining the type of the target cardiac section and whether it is a standard section based on this target information, this invention does not require a large amount of organ structure (e.g., cardiac structure) annotation process, making it easier to identify the section type of echocardiogram images. Furthermore, the echocardiogram section type recognition method provided by this invention has strong algorithm versatility and implements an end-to-end algorithm flow, which can better assist doctors in identifying section types and reduce the difficulty in the echocardiogram section type recognition process. In addition, by normalizing the output probability values ​​of the echocardiogram to be identified belonging to each preset section type output by the echocardiogram section type recognition model, this invention can obtain normalized probability values ​​of the echocardiogram to be identified belonging to other section types. Therefore, it solves the problem of identifying other section types without needing to specifically prepare data for other section types for training the echocardiogram section type recognition model.

[0155] It should be noted that, as those skilled in the art will understand, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0156] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0157] In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0158] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for identifying the type of echocardiographic image section, characterized in that, include: Acquire the echocardiogram image to be identified; The echocardiogram to be identified is preprocessed to remove the background region and obtain the region of interest image. The trained echocardiogram slice type recognition model is used to identify the region of interest image to obtain the output probability value of the echocardiogram image to be identified belonging to each preset slice type. The preset slice type includes a preset major slice type and a preset minor slice type corresponding to the preset major slice type. The minor slice type corresponding to the preset minor slice type is the standard slice of the major slice type corresponding to the preset major slice type. The output probability value of the echocardiogram to be identified belonging to each preset section type is normalized to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset section type and other section types, thereby obtaining the final identification result of the section type of the echocardiogram to be identified. The normalization process is performed on the output probability value of the echocardiogram to be identified belonging to each preset section type to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset section type and other section types, thereby obtaining the final identification result of the section type of the echocardiogram to be identified, including: The output probability values ​​of the echocardiogram to be identified belonging to each preset major section type are normalized to obtain the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types. Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types, the final identification result of the major section type of the echocardiogram to be identified is obtained, wherein the other major section types refer to major section types other than all preset major section types. The output probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type are normalized to obtain the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types. Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types, the final identification result of the sub-section type of the echocardiogram to be identified is obtained, wherein the other sub-section types represent non-standard sections. The normalization process is performed on the output probability values ​​of the echocardiogram to be identified belonging to each preset major section type to obtain the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types, including: The output probability values ​​of the echocardiogram image to be identified belonging to each preset major section type are normalized according to the following formula to obtain the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset major section type: In the formula, The output probability value is the number of preset major category sections for the echocardiogram image to be identified. The normalized probability value for the echocardiogram image to be identified to belong to the i-th preset major category of cross-section type. The threshold for determining whether the echocardiogram to be identified belongs to the i-th preset major category of cross-section type is used for the echocardiogram cross-section type recognition model, where N is the number of preset major category cross-section types, and i is a positive integer; Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type, the normalized probability values ​​of the echocardiogram to be identified belonging to other major section types are calculated.

2. The method for identifying the section type of echocardiogram according to claim 1, characterized in that, The echocardiogram section type recognition model includes a feature extraction network and a classification network. The feature extraction network is used to extract section features from the echocardiogram to be identified. The classification network includes several fully connected layers and several softmax layers. Each fully connected layer is connected to a softmax layer. The number of fully connected layers and the number of softmax layers are equal to the number of categories of the preset section type. The fully connected layers are used to perform nonlinear mapping regression on the extracted section features. The softmax layers are used to obtain the output probability value of the echocardiogram image to be identified belonging to the corresponding preset section type based on the result of the nonlinear mapping regression of the fully connected layers connected to them.

3. The method for identifying the section type of echocardiograms according to claim 1, characterized in that, Preprocessing the echocardiogram to be identified to remove background regions and obtain the region of interest image includes: Obtain the region of interest mask corresponding to the echocardiogram image to be identified, wherein in the region of interest mask, the pixel value of the background region is 0, and the pixel value of the region of interest is 1; The region of interest (ROI) image of the echocardiogram to be identified is multiplied with the region of interest mask to obtain the ROI image of the echocardiogram.

4. The method for identifying the section type of echocardiogram according to claim 3, characterized in that, The step of obtaining the mask of interest corresponding to the echocardiogram image to be identified includes: The pixels in all frames of the echocardiogram video corresponding to the echocardiogram image to be identified are analyzed. The region where the pixel value remains unchanged along the video time axis is taken as the background region, and the region outside the background region is taken as the region of interest, so as to obtain the region of interest mask corresponding to the echocardiogram video. The region of interest mask corresponding to the echocardiogram video is used as the region of interest mask corresponding to the echocardiogram image to be identified.

5. The method for identifying the section type of echocardiogram according to claim 1, characterized in that, The step of obtaining the final identification result of the major section type of the echocardiogram to be identified based on the normalized probability values ​​of the major section type to which the echocardiogram to be identified belongs and other major section types includes: Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset major section type and other major section types, the major section type with the largest normalized probability value is taken as the final identification result of the major section type of the echocardiogram to be identified.

6. The method for identifying the section type of echocardiogram according to claim 1, characterized in that, The normalization process for the output probability value of the echocardiogram to be identified belonging to each preset sub-section type, to obtain the normalized probability value of the echocardiogram to be identified belonging to each preset sub-section type and other sub-section types, includes: The output probability value of the echocardiogram image to be identified belonging to each preset sub-category of section is normalized according to the following formula to obtain the normalized probability value of the echocardiogram image to be identified belonging to each preset sub-category of section: In the formula, The output probability value is the number of preset sub-category sections for which the echocardiogram to be identified belongs. This is the normalized probability value of the echocardiogram image to be identified belonging to the i-th preset sub-category section type. The threshold for determining whether the echocardiogram to be identified belongs to the i-th preset sub-category of the echocardiogram is used by the echocardiogram section type recognition model, where N is the number of preset sub-category of the section type, and i is a positive integer; Based on the normalized probability values ​​of the echocardiogram to be identified belonging to each preset sub-section type, the normalized probability values ​​of the echocardiogram to be identified belonging to other sub-section types are calculated.

7. The method for identifying the section type of echocardiogram according to claim 1, characterized in that, The step of obtaining the final identification result of the sub-section type of the echocardiogram to be identified based on the normalized probability values ​​of the sub-section type to which the echocardiogram to be identified belongs and other sub-section types includes: Based on the normalized probability values ​​of the echocardiogram image to be identified belonging to each preset sub-section type and other sub-section types, the sub-section type with the largest normalized probability value is taken as the candidate identification result of the sub-section type of the echocardiogram image to be identified. Determine whether the major category of the candidate recognition result of the minor category of the cross-section is consistent with the final recognition result of the major category of the echocardiogram image to be recognized; If so, the candidate recognition result of the sub-section type is taken as the final recognition result of the sub-section type of the echocardiogram to be recognized.

8. The method for identifying the section type of echocardiogram according to claim 7, characterized in that, If the major category of the candidate recognition result of the minor category of the cross-section is inconsistent with the final recognition result of the major category of the echocardiogram image to be recognized, the recognition method further includes: Based on the final identification result of the major category section type of the echocardiogram to be identified, the preset minor category section type corresponding to the final identification result of the major category section type is taken as the target minor category section type. Determine whether the output probability value of the target sub-category section type is greater than or equal to the threshold value used by the echocardiogram section type recognition model to determine whether the echocardiogram to be identified belongs to the target sub-category section type; If so, the target sub-category section type is taken as the final identification result of the sub-category section type corresponding to the echocardiogram to be identified; If not, other sub-section types will be used as the final identification result of the sub-section type corresponding to the echocardiogram image to be identified.

9. An electronic device, characterized in that, The method includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the method for identifying the section type of echocardiogram as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements the method for identifying the cross-sectional type of echocardiograms as described in any one of claims 1 to 8.

Images