Ejecting function evaluation value method and device based on semi-supervised depth segmentation model

Abstract

The invention provides an ejection function evaluation value method and device based on a semi-supervised depth segmentation model, and can be applied to the technical field of medical detection. The ejection function evaluation value method based on the semi-supervised depth segmentation model comprises the following steps: acquiring an echocardiography video; performing segmentation processing on the echocardiography video by using a semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point in a left ventricular region in each frame of echocardiography image; according to the plurality of relative depth estimation values in each frame of echocardiographic image, calculating to obtain a candidate volume of a left ventricle in each frame of echocardiographic image; based on a first preset rule, multiple target systolic volumes and multiple target diastolic volumes are determined from the multiple candidate volumes, and according to the multiple target systolic volumes and the multiple target diastolic volumes, a target ejection function evaluation value of the left ventricle is obtained.

Description

Technical Field

[0001] The present invention relates to the field of medical detection technology, and more specifically to a method and device for evaluating ejection function based on a semi-supervised deep segmentation model. Background Art

[0002] Since the acquired echocardiogram of the object is real-time, and the process of acquiring the echocardiogram of the object through ultrasound equipment has the advantages of low cost and no radiation, it is currently a common detection method to use non-invasive echocardiography as an intermediate result to conduct targeted diagnosis of the above-mentioned object. Furthermore, left ventricular ejection fraction is also one of the important data used to help diagnose the pumping function of the above-mentioned object. However, there is a large error between the existing left ventricular ejection fraction predicted by echocardiography or echocardiography video and the actual result.

[0003] In the process of realizing the above-mentioned inventive concept, it was found that: in the related technology, since it is impossible to accurately process the ultrasonic cardiology video to obtain the end-diastolic frame and the end-systolic frame in the ultrasonic cardiology video, it is impossible to obtain an accurate left ventricular ejection fraction, and when the end-diastolic frame and the end-systolic frame are processed to estimate the left ventricular ejection fraction, it is impossible to intuitively see the intermediate estimation process and the intermediate estimation results, so that the estimation process cannot be regulated, resulting in technical problems such as large error in the left ventricular ejection fraction and poor flexibility. Summary of the invention

[0004] In view of the above problems, the present invention provides a method and device for evaluating the ejection function value based on a semi-supervised deep segmentation model.

[0005] According to a first aspect of the present invention, a method for evaluating an ejection function based on a semi-supervised deep segmentation model is provided, comprising: acquiring an ultrasonic video, wherein the ultrasonic video comprises a plurality of frames of ultrasonic images; performing segmentation processing on the ultrasonic video using a semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel located in a left ventricular region in each frame of the ultrasonic image, wherein the semi-supervised deep segmentation model comprises a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the depth of each pixel located in the left ventricular region; calculating a candidate volume of the left ventricle in each frame of the ultrasonic image according to a plurality of relative depth estimation values ​​in each frame of the ultrasonic image; determining a plurality of target systolic volumes and a plurality of target diastolic volumes from the plurality of candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes, wherein the first predetermined rule represents that a candidate volume corresponding to a minimum value among the plurality of candidate volumes is a target systolic volume, and a candidate volume corresponding to a maximum value among the plurality of candidate volumes is a target diastolic volume.

[0006] According to an embodiment of the present invention, an ultrasonic video is segmented using a semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of the ultrasonic image, including: for any frame of the ultrasonic image in multiple frames of ultrasonic images, any frame of the ultrasonic image is input into the semi-supervised depth segmentation model to calculate the depth value of each pixel point using the semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point; according to a second predetermined rule, a relative depth estimation value corresponding to each pixel point located in the left ventricular region is screened from multiple relative depth estimation values; the above operation is performed on each frame of the ultrasonic image in the multiple frames of the ultrasonic image to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of the ultrasonic image.

[0007] According to an embodiment of the present invention, the second predetermined rule characterizes that when the relative depth estimation value is 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in an area outside the left ventricle; when the relative depth estimation value is not 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in an area inside the left ventricle.

[0008] According to an embodiment of the present invention, a candidate volume of the left ventricle in each frame of the ultrasonic cardiac image is calculated based on multiple relative depth estimation values ​​in each frame of the ultrasonic cardiac image, including: summing the relative depth estimation value of each pixel point in each frame of the ultrasonic cardiac image to calculate the candidate volume of the left ventricle in each frame of the ultrasonic cardiac image.

[0009] According to an embodiment of the present invention, based on a first predetermined rule, multiple target systolic volumes and multiple target diastolic volumes are determined from multiple candidate volumes, and a target ejection function evaluation value of the left ventricle is obtained according to the multiple target systolic volumes and the multiple target diastolic volumes, including: generating an initial volume curve graph of the left ventricle according to the multiple candidate volumes; performing peak-finding processing on the initial volume curve graph by using a peak-finding function to determine multiple target systolic volumes and multiple target diastolic volumes; and calculating the target ejection function evaluation value according to the multiple target systolic volumes and the multiple target diastolic volumes.

[0010] According to an embodiment of the present invention, a target ejection function evaluation value is calculated based on multiple target systolic volumes and multiple target diastolic volumes, including: based on a cardiac cycle division rule, performing cardiac cycle division processing on an initial volume curve graph to obtain a target volume curve graph, wherein the target volume curve graph represents a curve graph divided into multiple cardiac cycles, and the cardiac cycle division rule represents division with an adjacent target systolic volume and a target diastolic volume as one cardiac cycle; performing ejection calculation on the target systolic volume and the target diastolic volume in each cardiac cycle to obtain an ejection fraction corresponding to each cardiac cycle; and averaging multiple ejection fractions to obtain a target ejection function evaluation value.

[0011] According to an embodiment of the present invention, a semi-supervised deep segmentation model is trained in the following manner, including: obtaining a training data set, wherein the training data set includes an ultrasonic cardiology sample video, the ultrasonic cardiology sample video is composed of multiple frames of ultrasonic cardiology sample images, and the multiple frames of ultrasonic cardiology sample images include diastolic sample images containing segmentation labels, contraction sample images containing segmentation labels, and ultrasonic cardiology sample images without segmentation labels; performing pre-segmentation processing on the ultrasonic cardiology sample video to obtain a pre-depth segmentation image corresponding to each frame of the ultrasonic cardiology sample image in the ultrasonic cardiology sample video, wherein the pre-depth segmentation image includes a pre-depth segmentation label corresponding to each pixel point, and the pre-depth segmentation label represents the depth of each pixel point in the left ventricular area; The ultrasonic cardiogram sample video is input into the semi-supervised deep segmentation model to be trained for segmentation processing, and a segmentation training image corresponding to each frame of the ultrasonic cardiogram sample image in the ultrasonic cardiogram sample video is obtained, wherein the semi-supervised deep segmentation model to be trained is composed of a first image segmentation network and a second image segmentation network with the same initial weights, and the segmentation training images include diastole training images, contraction training images and multiple frames of cardiac training images; based on the loss function, a training loss value is obtained according to the multiple frames of pre-deep segmentation images, diastole training images, contraction training images and multiple frames of cardiac training images; according to the training loss value, the trained semi-supervised deep segmentation model is obtained by adjusting the initial weights of the first image segmentation network and the initial weights of the second image segmentation network.

[0012] According to an embodiment of the present invention, a diastolic training image includes a first diastolic training image processed by a first image segmentation network and a second diastolic training image processed by a second image segmentation network, a contraction training image includes a first contraction training image processed by a first image segmentation network and a second contraction training image processed by a second image segmentation network, the first diastolic training image and the second diastolic training image include diastolic pixels, and the first contraction training image and the second contraction training image include contraction pixels; based on a loss function, a training loss value is obtained according to multiple frames of pre-depth segmentation images, diastolic training images, contraction training images and multiple frames of cardiac training images, including: for each diastolic pixel in the first diastolic training image, according to the training depth of the diastolic pixel and the pre-depth segmentation label of the first target pixel, the diastolic loss of each of the multiple diastolic pixels in the first diastolic training image is obtained. Function, wherein the first target pixel point is determined from the first pre-depth segmentation image corresponding to the diastolic training image according to the coordinate information of the diastolic pixel point; for each contraction pixel point in the second contraction training image, according to the training depth of the contraction pixel point and the pre-depth segmentation label of the second target pixel point, the contraction loss function of each of the plurality of contraction pixels in the second contraction training image is obtained, wherein the second target pixel point is determined from the second pre-depth segmentation image corresponding to the contraction training image according to the coordinate information of the contraction pixel point; the training loss value is determined according to the supervision loss value determined based on the plurality of diastolic loss functions, the plurality of contraction loss functions, the size information of the diastolic training image and the size information of the contraction training image, and the target cross pseudo-supervisory loss value determined based on the size information of the multi-frame cardiac training image, the diastolic training image, the contraction training image and the multi-frame pre-depth segmentation image.

[0013] According to an embodiment of the present invention, a cardiac training image includes a first network training image processed by a first image segmentation network and a second network training image processed by a second image segmentation network, the first network training image includes a first cardiac pixel point, and the second network training image includes a second cardiac pixel point; the target cross-pseudo-supervision loss value is determined in the following manner: according to the training depth of each first cardiac pixel point and the training depth of the third target pixel point, a first mean square error function of each of the first cardiac pixels in the plurality of first network training images is determined, wherein the third target pixel point is determined from the second network training image according to the coordinate information of the first cardiac pixel point; according to the training depth of each second cardiac pixel point and the training depth of the fourth target pixel point, a second mean square error function of each of the second cardiac pixels in the plurality of second network training images is determined, wherein the fourth target pixel point is determined from the second cardiac pixel point according to the coordinate information of the second cardiac pixel point. The first network training image is determined; the diastolic training image and the systolic training image are processed to obtain a third mean square error function of each of the diastolic pixels in the first diastolic training image, a fourth mean square error function of each of the diastolic pixels in the second diastolic training image, a fifth mean square error function of each of the systolic pixels in the first systolic training image, and a sixth mean square error function of each of the systolic pixels in the second systolic training image; based on the multiple first mean square error functions, the multiple second mean square error functions, and the size information of the multiple frames of cardiac training images, a first cross pseudo-supervised loss value is determined; based on the multiple third mean square error functions, the multiple fourth mean square error functions, the multiple fifth mean square error functions, the multiple sixth mean square error functions, the size information of the diastolic training images, and the size information of the systolic training images, a second cross pseudo-supervised loss value is determined; and according to the first cross pseudo-supervised loss value and the second cross pseudo-supervised loss value, a target cross pseudo-supervised loss value is determined.

[0014] The second aspect of the present invention provides an apparatus for evaluating ejection function based on a semi-supervised deep segmentation model, characterized in that it includes: an acquisition module for acquiring an ultrasonic cardiology video, wherein the ultrasonic cardiology video includes multiple frames of ultrasonic cardiology images; a segmentation module for segmenting the ultrasonic cardiology video using the semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of the ultrasonic cardiology image, wherein the semi-supervised deep segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the relative depth of each pixel point located in the left ventricular region. a depth in a left ventricular region; a calculation module, for calculating a candidate volume of the left ventricle in each frame of the ultrasound cardiogram according to a plurality of relative depth estimation values ​​in each frame of the ultrasound cardiogram; an obtaining module, for determining a plurality of target contraction volumes and a plurality of target diastolic volumes from the plurality of candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target contraction volumes and the plurality of target diastolic volumes, wherein the first predetermined rule characterizes that a candidate volume corresponding to a minimum value among the plurality of candidate volumes is a target contraction volume, and a candidate volume corresponding to a maximum value among the plurality of candidate volumes is a target diastolic volume.

[0015] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0016] The fourth aspect of the present invention also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above method.

[0017] The fifth aspect of the present invention also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0018] According to the method and device for evaluating ejection function based on a semi-supervised deep segmentation model of the present invention, an ultrasonic video is acquired, and the ultrasonic video is segmented using a semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel point in the left ventricular region in each frame of the ultrasonic image. According to multiple relative depth estimation values ​​in each frame of the ultrasonic image, the candidate volume of the left ventricle in each frame of the ultrasonic image is calculated, and based on the first predetermined rule, multiple target systolic volumes and multiple target diastolic volumes are determined from multiple candidate volumes, and the target ejection function evaluation value of the left ventricle is obtained according to the multiple target systolic volumes and multiple target diastolic volumes. The method achieves obtaining the accurate target ejection function evaluation value of the left ventricle of the target object according to the ultrasonic video using a pre-trained model, and determining the target diastolic volume and the target systolic volume based on the maximum and minimum values ​​in the candidate volumes, thereby accurately locating the end-diastolic frame and the end-systolic frame in the ultrasonic video, so as to obtain a highly accurate and robust left ventricular ejection fraction. In the process of obtaining the target ejection function evaluation value of the left ventricle of the target object, relevant professionals can call up the semi-supervised deep segmentation model at any time during the operation of the model to obtain parameters such as the relative depth estimation value of each pixel point in any frame of ultrasound cardiac image, candidate volumes of multiple left ventricles, end-diastolic frame images and end-systolic frame images, so as to facilitate relevant professionals to regulate the semi-supervised deep segmentation model, visualize the intermediate process, and improve the convenience and interactivity of operation.

[0019] According to an embodiment of the present invention, further, a semi-supervised deep segmentation model is built according to the first image segmentation network and the second image segmentation network, and then a large amount of model training is performed on the built semi-supervised deep segmentation model to obtain a trained semi-supervised deep segmentation model. When the left ventricular ejection fraction of the target object is estimated, the semi-supervised deep segmentation model is directly used to process the ultrasound cardiology video to obtain the target ejection function evaluation value of the left ventricle. The use of a highly robust semi-supervised deep segmentation model to process ultrasound cardiology videos is achieved, thereby improving work efficiency and reducing work costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0021] Figure 1 An application scenario diagram of a method for evaluating ejection function based on a semi-supervised deep segmentation model according to an embodiment of the present invention is shown;

[0022] Figure 2 A flow chart showing a method for evaluating ejection function based on a semi-supervised deep segmentation model according to an embodiment of the present invention is shown;

[0023] Figure 3 A schematic diagram showing a left ventricle segmentation label in an ultrasound cardiography video according to an embodiment of the present invention is shown;

[0024] Figure 4a A schematic diagram showing the regional heat of the left ventricle after segmentation using a semi-supervised deep segmentation model according to an embodiment of the present invention is shown;

[0025] Figure 4b A schematic diagram showing a regional heat curve of the left ventricle after segmentation using a semi-supervised deep segmentation model according to an embodiment of the present invention is shown;

[0026] Figure 5 A schematic diagram showing a process of obtaining a supervision loss value according to an embodiment of the present invention;

[0027] Figure 6 A schematic diagram showing a process of obtaining a cross pseudo-supervision loss value according to an embodiment of the present invention is shown;

[0028] Figure 7 A schematic diagram showing a comparison between the ejection fraction obtained by using the model of the present invention and the true value according to an embodiment of the present invention;

[0029] Figure 8 A structural block diagram of a device for evaluating ejection function of a semi-supervised deep segmentation model according to an embodiment of the present invention is shown;

[0030] Fig. 9 A block diagram of an electronic device showing a method for evaluating ejection function based on a semi-supervised deep segmentation model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0032] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0033] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0034] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0035] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0036] Echocardiography is a non-invasive bio-imaging examination method that uses ultrasonic technology to observe the structure of the target, such as the structure of the heart, so as to assist in the targeted diagnosis of the target as an intermediate result. Due to the advantages of high real-time, low cost and no radiation, echocardiography is widely used to assist in the evaluation of the target's operating function. However, due to the limitations of echocardiography such as noise and contrast, it is impossible to obtain accurate left ventricular volume, which makes it impossible to assist in the targeted diagnosis of the target. At the same time, ventricular ejection fraction is one of the important auxiliary judgment indicators of cardiac pumping function in clinical reports, and the process of estimating left ventricular volume based on echocardiography is very time-consuming. Therefore, the existing technology obtains the left ventricular ejection fraction by processing the echocardiography video.

[0037] The existing measurement of left ventricular ejection fraction is usually to find the left ventricular end-diastolic frame and end-systolic frame by independently segmenting the left ventricular region of each frame, and to estimate the end-diastolic volume and end-systolic volume according to the Simpson method, so as to estimate the left ventricular ejection fraction. However, the existing method of obtaining the left ventricular ejection fraction based on the ultrasound cardiography video can neither regulate the specific operation nor obtain the target situation in the prediction process, so it is impossible to judge the accuracy of the obtained left ventricular ejection fraction, and it cannot be widely used in actual operation. During the research and development process, the researchers found that in the related technology, it is impossible to obtain the accurate left ventricular ejection fraction because the end-diastolic frame and the end-systolic frame in the ultrasound cardiography video cannot be accurately obtained. In addition, when estimating the left ventricular ejection fraction, it is impossible to intuitively see the intermediate estimation process and the intermediate estimation results, so it is impossible to regulate the estimation process, resulting in technical problems such as large error and poor flexibility of the left ventricular ejection fraction.

[0038] In view of this, an embodiment of the present invention provides a method for evaluating ejection function based on a semi-supervised deep segmentation model, comprising: obtaining an ultrasonic video, wherein the ultrasonic video includes multiple frames of ultrasonic images; using the semi-supervised deep segmentation model to segment the ultrasonic video to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of the ultrasonic image, wherein the semi-supervised deep segmentation model includes a first image segmentation convolutional neural network and a second image segmentation convolutional neural network with different weights, and the relative depth estimation value represents the relative depth of each pixel point located in the left ventricular region. The method comprises the steps of: determining a depth in a left ventricular region; calculating a candidate volume of the left ventricle in each frame of the ultrasound cardiogram according to a plurality of relative depth estimation values ​​in each frame of the ultrasound cardiogram; determining a plurality of target systolic volumes and a plurality of target diastolic volumes from the plurality of candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes, wherein the first predetermined rule characterizes that a candidate volume corresponding to a minimum value among the plurality of candidate volumes is a target systolic volume, and a candidate volume corresponding to a maximum value among the plurality of candidate volumes is a target diastolic volume.

[0039] Figure 1 A diagram showing an application scenario of a method for evaluating ejection function based on a semi-supervised deep segmentation model according to an embodiment of the present invention.

[0040] like Figure 1 As shown, the application scenario according to this embodiment may include a first ultrasound device 101, a target object 102, and a receiver 103. The first ultrasound device 101 is used to send an ultrasound signal to the target object 102.

[0041] The user may use the first ultrasound device 101 to interact with the target object 102 and the receiver 103 to receive or send signals, etc.

[0042] The receiver 103 may be a receiver for receiving various ultrasonic signals, for example, receiving and processing the ultrasonic signal sent by the first ultrasonic device 101 (for example only). The receiver 103 may analyze and process the received ultrasonic signal and other data, and feed back the processing result (for example, a web page, information, or data obtained or generated according to a user request) to the terminal device.

[0043] It should be noted that the method for evaluating the ejection function value based on the semi-supervised deep segmentation model provided in the embodiment of the present invention can generally be executed by the receiver 103. Accordingly, the device for evaluating the ejection function value based on the semi-supervised deep segmentation model provided in the embodiment of the present invention can generally be arranged in the receiver 103. The method for evaluating the ejection function value based on the semi-supervised deep segmentation model provided in the embodiment of the present invention can also be executed by a receiver or a receiver cluster that is different from the receiver 103 and can communicate with the first ultrasound device 101 and / or the receiver 103. Accordingly, the device for evaluating the ejection function value based on the semi-supervised deep segmentation model provided in the embodiment of the present invention can also be arranged in a receiver or a receiver cluster that is different from the receiver 103 and can communicate with the first ultrasound device 101 and / or the receiver 103.

[0044] It should be understood that Figure 1 The number of the first ultrasound device, the target object and the receiver in the embodiment is only for illustration. According to the implementation requirements, there may be any number of ultrasound devices, observation objects and receivers.

[0045] The following will be based on Figure 1 The scene described by Figure 2~Figure 7 The method for evaluating the ejection function value based on the semi-supervised deep segmentation model of an embodiment of the present invention is described in detail.

[0046] Figure 2 A flowchart of a method for evaluating ejection function based on a semi-supervised deep segmentation model according to an embodiment of the present invention is shown.

[0047] like Figure 2 As shown, the method for evaluating the ejection function value based on the semi-supervised deep segmentation model of this embodiment includes operations S210 to S240.

[0048] In operation S210 , an ultrasound cardiography video is acquired.

[0049] According to an embodiment of the present invention, the ultrasound video includes a plurality of ultrasound image frames.

[0050] According to an embodiment of the present invention, each frame of the acquired target ultrasonic cardiac image includes the region of the apical four chambers, and the apical four chambers include the left ventricle region, the left atrium region, the right ventricle region and the right atrium region.

[0051] According to the embodiment of the present invention, each frame of the ultrasound cardiology image in the ultrasound cardiology video can be extracted separately for review.

[0052] In operation S220, the ultrasonic video is segmented using the semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricle region in each frame of the ultrasonic image.

[0053] According to an embodiment of the present invention, the semi-supervised deep segmentation model includes a first image segmentation network and a second image segmentation network with different weights.

[0054] According to an embodiment of the present invention, the relative depth estimation value represents the depth of each pixel point in the left ventricle region.

[0055] According to an embodiment of the present invention, an untrained semi-supervised depth segmentation model is pre-trained to obtain a trained semi-supervised depth segmentation model, and the trained semi-supervised depth segmentation model is used to calculate the segmentation depth of each frame of the acquired ultrasonic cardiogram video to obtain a relative depth estimation value corresponding to each pixel point in each frame of the ultrasonic cardiogram. According to the depth value corresponding to each pixel point, it can be determined whether the pixel point is located in the left ventricular region, thereby obtaining a relative depth estimation value corresponding to each pixel point located in the left ventricular region.

[0056] According to an embodiment of the present invention, each frame of ultrasound cardiac image containing a relative depth estimation value corresponding to each pixel point can be extracted separately for review, so that relevant professionals can check the depth value of any pixel point to adjust the model.

[0057] In operation S230, a candidate volume of the left ventricle in each frame of the ultrasound image is calculated according to a plurality of relative depth estimation values ​​in each frame of the ultrasound image.

[0058] According to an embodiment of the present invention, the candidate volume of the left ventricle of each frame of the ultrasound cardiology image is obtained by calculation, so as to calculate the target left ventricular ejection fraction, that is, the target ejection function evaluation value.

[0059] According to the embodiment of the present invention, the candidate volume of the left ventricle in each frame of ultrasound cardiac image can be extracted separately for review.

[0060] In operation S240, a plurality of target systolic volumes and a plurality of target diastolic volumes are determined from a plurality of candidate volumes based on a first predetermined rule, and a target ejection function evaluation value of the left ventricle is obtained according to the plurality of target systolic volumes and the plurality of target diastolic volumes.

[0061] According to an embodiment of the present invention, the first predetermined rule indicates that the candidate volume corresponding to the minimum value among the multiple candidate volumes is the target contraction volume, and the candidate volume corresponding to the maximum value among the multiple candidate volumes is the target diastolic volume.

[0062] According to an embodiment of the present invention, the target ejection function assessment value of the left ventricle may be characterized as a target ejection fraction of the left ventricle.

[0063] According to an embodiment of the present invention, a volume curve graph corresponding to the left ventricle can be generated based on multiple candidate volumes, and multiple minimum points and maximum points can be determined from the curve graph based on a first predetermined rule, and the multiple minimum points can be determined as multiple target contraction volumes, and multiple ultrasound cardiac images corresponding to the multiple minimum points can be determined as target end-systolic frames, and the multiple maximum points can be determined as multiple target diastolic volumes, and multiple ultrasound cardiac images corresponding to the multiple maximum points can be determined as target end-diastolic frames.

[0064] According to an embodiment of the present invention, the obtained target ejection function evaluation value of the left ventricle can help relevant personnel to judge the relevant functions of the target.

[0065] For example, an ultrasonic test is performed on the heart of the target object to obtain an ultrasonic cardiogram video, and the ultrasonic cardiogram video is segmented using a trained semi-supervised deep segmentation model to obtain a depth value corresponding to each pixel in the left ventricle region of each frame of the ultrasonic cardiogram, that is, a relative depth estimation value. A candidate volume of the left ventricle in each frame of the ultrasonic cardiogram is calculated based on the relative depth estimation value of each pixel in each frame of the ultrasonic cardiogram, and multiple target systolic volumes and multiple target diastolic volumes are selected from the multiple candidate volumes. A target ejection function evaluation value of the left ventricle in the heart of the target object is calculated based on the multiple target systolic volumes and the multiple target diastolic volumes, so that scientists or professionals in related fields can identify the cardiac function of the target object based on the target ejection function evaluation value of the left ventricle, thereby making a judgment appropriate for the target object.

[0066] According to the embodiment of the present invention, each target diastolic volume, target systolic volume, end-systolic frame image, and end-diastolic frame image can be extracted separately for review.

[0067] According to an embodiment of the present invention, an ultrasound video is acquired, and the ultrasound video is segmented using a semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel point in the left ventricular region in each frame of the ultrasound image. Based on multiple relative depth estimation values ​​in each frame of the ultrasound image, a candidate volume of the left ventricle in each frame of the ultrasound image is calculated, and based on a first predetermined rule, multiple target systolic volumes and multiple target diastolic volumes are determined from multiple candidate volumes, and a target ejection function evaluation value of the left ventricle is obtained based on the multiple target systolic volumes and multiple target diastolic volumes. The accurate target ejection function evaluation value of the left ventricle of the target object is obtained based on the ultrasound video using a pre-trained model, and the target diastolic volume and target systolic volume are determined based on the maximum and minimum values ​​in the candidate volumes, thereby accurately locating the end-diastolic frame and the end-systolic frame in the ultrasound video, so as to obtain a highly accurate and robust left ventricular ejection fraction. In the process of obtaining the target ejection function evaluation value of the left ventricle of the target object, relevant professionals can call up the semi-supervised deep segmentation model at any time during the operation of the model to obtain parameters such as the relative depth estimation value of each pixel point in any frame of ultrasound cardiac image, candidate volumes of multiple left ventricles, end-diastolic frame images and end-systolic frame images, so as to facilitate relevant professionals to regulate the semi-supervised deep segmentation model, visualize the intermediate process, and improve the convenience and interactivity of operation.

[0068] According to an embodiment of the present invention, further, a semi-supervised deep segmentation model is built according to the first image segmentation network and the second image segmentation network, and then a large amount of model training is performed on the built semi-supervised deep segmentation model to obtain a trained semi-supervised deep segmentation model. When the left ventricular ejection fraction of the target object is estimated, the semi-supervised deep segmentation model is directly used to process the ultrasound cardiology video to obtain the target ejection function evaluation value of the left ventricle. The use of a highly robust semi-supervised deep segmentation model to process ultrasound cardiology videos is achieved, thereby improving work efficiency and reducing work costs.

[0069] It should be noted that the target ejection function evaluation value of the left ventricle obtained by the present invention is only an intermediate result, and the diagnosis result or health status cannot be directly derived from the target ejection function evaluation value of the left ventricle obtained by the method of the present invention.

[0070] According to an embodiment of the present invention, a method for segmenting an ultrasound video using a semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel in the left ventricle region in each frame of the ultrasound image includes the following operations.

[0071] According to an embodiment of the present invention, for any frame of ultrasound cardiac images in multiple frames of ultrasound cardiac images, any frame of ultrasound cardiac images is input into a semi-supervised depth segmentation model, so as to calculate the depth value of each pixel point using the semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point.

[0072] According to an embodiment of the present invention, according to a second predetermined rule, a relative depth estimation value corresponding to each pixel point located in the left ventricle region is obtained by screening out a plurality of relative depth estimation values.

[0073] According to an embodiment of the present invention, the second predetermined rule characterizes that when the relative depth estimation value is 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in an area outside the left ventricle; when the relative depth estimation value is not 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in an area inside the left ventricle.

[0074] For example, when the relative depth estimation values ​​of the pixels in the regions [20:20], [20:40], [40:20] and [40:40] of the ultrasonic cardiac image are all greater than 0, that is, when the relative depth estimation values ​​are not 0, it is confirmed that all the pixels in the region are located in the left ventricle region, and the relative depth estimation values ​​corresponding to all the pixels in the left ventricle region are obtained.

[0075] According to an embodiment of the present invention, the above operation is performed on each frame of the multiple frames of ultrasonic cardiac images to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricle region in each frame of the ultrasonic cardiac image.

[0076] According to an embodiment of the present invention, a relative depth estimation value of each pixel is first obtained for each frame of ultrasound cardiology image in the ultrasound cardiology video input into the semi-supervised depth segmentation model, and then, based on whether the relative depth estimation value is greater than 0, it is judged whether the pixel is located in the left ventricular region, thereby obtaining the depth value of each pixel located in the left ventricular region in each frame of ultrasound cardiology image, that is, the relative depth estimation value.

[0077] According to an embodiment of the present invention, an ultrasonic video containing multiple frames of ultrasonic images is input into a semi-supervised deep segmentation model, and the semi-supervised deep segmentation model calculates the depth value of each pixel in each frame of the ultrasonic image to obtain a relative depth estimation value corresponding to each pixel. Based on a second predetermined rule, it is determined whether the relative depth estimation value is equal to 0 according to the relative depth estimation value. When the relative depth estimation value is not equal to 0, it is confirmed that the pixel corresponding to the relative depth estimation value is located in the left ventricular region, thereby obtaining a relative depth estimation value corresponding to each pixel located in the left ventricular region in each frame of the ultrasonic image. This realizes the division of regions for each frame of the ultrasonic video using a semi-supervised deep segmentation model with high robustness, clearly and accurately obtains the region of the left ventricle in each frame of the ultrasonic image, and obtains the relative depth estimation value corresponding to each pixel located in the left ventricular region, so that the volume of the left ventricle is calculated according to the relative depth estimation value of the pixel points in the left ventricular region, thereby improving the estimation efficiency.

[0078] According to an embodiment of the present invention, the target electromagnetic echo signal includes an electromagnetic echo signal corresponding to each space point in the three-dimensional space.

[0079] According to an embodiment of the present invention, a method for calculating a candidate volume of a left ventricle in each frame of an ultrasound image according to a plurality of relative depth estimation values ​​in each frame of an ultrasound image includes the following operations.

[0080] According to an embodiment of the present invention, the relative depth estimation value of each pixel in each frame of the ultrasonic cardiogram is summed up to calculate the candidate volume of the left ventricle in each frame of the ultrasonic cardiogram.

[0081] For example, the left ventricle region of the ultrasonic cardiogram includes 6 pixels, the relative depth estimation value corresponding to the first pixel is 2, the relative depth estimation value corresponding to the second pixel is 13, the relative depth estimation value corresponding to the third pixel is 9, the relative depth estimation value corresponding to the fourth pixel is 4, the relative depth estimation value corresponding to the fifth pixel is 0.71, and the relative depth estimation value corresponding to the sixth pixel is 0.8. By summing the above 6 relative depth estimation values, the candidate volume of the left ventricle of the frame of the ultrasonic cardiogram can be 29.51.

[0082] According to an embodiment of the present invention, by summing the relative depth estimation values ​​of each pixel in each frame of the ultrasonic cardiogram, the candidate volume of the left ventricle in each frame of the ultrasonic cardiogram can be obtained, thereby achieving accurate calculation of the volume of the left ventricle in each frame of the ultrasonic cardiogram in the video, so as to obtain an accurate ejection fraction of the left ventricle based on the candidate volume of the left ventricle.

[0083] According to an embodiment of the present invention, based on a first predetermined rule, a plurality of target systolic volumes and a plurality of target diastolic volumes are determined from a plurality of candidate volumes, and a method for obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes includes the following operations.

[0084] According to an embodiment of the present invention, an initial volume curve map of the left ventricle is generated based on a plurality of candidate volumes.

[0085] According to an embodiment of the present invention, each frame of the ultrasound cardiology image corresponds to a candidate volume of the left ventricle, so that the ultrasound cardiology video corresponds to multiple candidate volumes, and an initial volume curve graph corresponding to the left ventricle is generated according to the multiple candidate volumes. The initial volume curve graph includes multiple minimum points, multiple maximum points and other numerical points, wherein the minimum points and the maximum points can correspond to the target contraction and target relaxation states, for example, the movement states of cardiac contraction and cardiac diastole.

[0086] According to an embodiment of the present invention, a peak-finding function is used to perform peak-finding processing on the initial volume curve graph to determine a plurality of target contraction volumes and a plurality of target diastolic volumes.

[0087] According to an embodiment of the present invention, a plurality of minimum points and a plurality of maximum points are captured from an initial volume curve graph using a peak-finding function, and a candidate volume of the minimum point is confirmed to be a target contraction volume, and an ultrasound cardiac image corresponding to the candidate volume of the minimum point is an end-systolic frame, and a candidate volume of the maximum point is a target diastolic volume, and an ultrasound cardiac image corresponding to the candidate volume of the maximum point is an end-diastolic frame, thereby obtaining a plurality of target contraction volumes and a plurality of target diastolic volumes.

[0088] According to an embodiment of the present invention, a target ejection function evaluation value is calculated based on a plurality of target systolic volumes and a plurality of target diastolic volumes.

[0089] According to an embodiment of the present invention, the heartbeat, i.e., the cardiac cycle, can be divided according to multiple target systolic volumes and multiple target diastolic volumes. According to the target systolic volume and target diastolic volume in each cardiac cycle, the target ejection function evaluation value of the target left ventricle can be obtained, so as to serve as an intermediate result to assist relevant professionals in conducting targeted diagnosis.

[0090] According to an embodiment of the present invention, an initial volume curve of the left ventricle is constructed based on multiple candidate volumes of the left ventricle, and multiple minimum points are determined from the initial volume curve as target systolic volume and target diastolic volume using a peak-finding algorithm, and multiple end-systolic frames corresponding to the target systolic volume and end-diastolic frames corresponding to the target diastolic volume can be determined at the same time. The target ejection function evaluation value of the target left ventricle can be calculated based on the multiple target systolic volumes and multiple target diastolic volumes. The candidate volume of the left ventricle obtained based on the semi-supervised deep segmentation model is used to obtain an accurate target ejection function evaluation value of the left ventricle. In addition, while obtaining the target ejection function evaluation value of the left ventricle, parameters such as the initial volume curve of the left ventricle, the end-diastolic frame, the end-systolic frame, the target systolic volume, and the target diastolic volume can be extracted and consulted separately, so that relevant personnel can verify and timely consult and adjust any value in the intermediate process.

[0091] According to an embodiment of the present invention, a method for calculating a target ejection function evaluation value according to a plurality of target systolic volumes and a plurality of target diastolic volumes includes the following operations.

[0092] According to an embodiment of the present invention, based on a cardiac cycle division rule, the initial volume curve graph is processed by cardiac cycle division to obtain a target volume curve graph.

[0093] According to an embodiment of the present invention, the target volume curve graph represents a curve graph divided into a plurality of cardiac cycles, and the cardiac cycle division rule represents division of an adjacent target systolic volume and a target diastolic volume as one cardiac cycle.

[0094] According to an embodiment of the present invention, the cardiac cycle division rule is characterized by dividing a cardiac cycle by an adjacent target systolic volume and a target diastolic volume, that is, each cardiac cycle includes an end-diastolic frame and an end-systolic frame.

[0095] According to an embodiment of the present invention, ejection calculation is performed on the target systolic volume and the target diastolic volume in each cardiac cycle to obtain the ejection fraction corresponding to each cardiac cycle.

[0096] According to an embodiment of the present invention, the ejection fraction can be calculated according to formula (1).

[0097] (1)

[0098] Among them, EF can be represented as the ejection fraction corresponding to one cardiac cycle, EDV can be represented as the target diastolic volume within one cardiac cycle, and ESV can be represented as the target systolic volume within one cardiac cycle.

[0099] For example, according to the cardiac cycle division rule, the initial volume curve graph is divided into 6 cardiac beats, that is, 6 cardiac cycles, the target contraction volume in the first cardiac cycle is 1L, and the target diastolic volume is 3L, the target contraction volume in the second cardiac cycle is 0.5L, and the target diastolic volume is 3.5L, the target contraction volume in the third cardiac cycle is 1L, and the target diastolic volume is 4L, the target contraction volume in the fourth cardiac cycle is 2L, and the target diastolic volume is 3L, and the target contraction volume in the fifth cardiac cycle is 0.5L, and the target diastolic volume is 3.5L. The target systolic volume is 2L, the target diastolic volume is 4L, the target contraction volume in the sixth cardiac cycle is 2.5L, and the target diastolic volume is 3.5L. The ejection fraction corresponding to the first cardiac cycle is 66.7%, the ejection fraction corresponding to the second cardiac cycle is 85.7%, the ejection fraction corresponding to the third cardiac cycle is 75%, the ejection fraction corresponding to the fourth cardiac cycle is 33.3%, the ejection fraction corresponding to the fifth cardiac cycle is 50%, and the ejection fraction corresponding to the sixth cardiac cycle is 28.6%.

[0100] According to an embodiment of the present invention, a plurality of ejection fractions are averaged to obtain a target ejection function evaluation value.

[0101] For example, the ejection fraction corresponding to the first cardiac cycle is 66.7%, the ejection fraction corresponding to the second cardiac cycle is 85.7%, the ejection fraction corresponding to the third cardiac cycle is 75%, the ejection fraction corresponding to the fourth cardiac cycle is 33.3%, the ejection fraction corresponding to the fifth cardiac cycle is 50%, and the ejection fraction corresponding to the sixth cardiac cycle is 28.6%. The target ejection function assessment value of the left ventricle of the target subject is 56.55%.

[0102] According to an embodiment of the present invention, the ejection fractions corresponding to any number of cardiac cycles may be randomly selected from multiple cardiac cycles for average calculation to obtain a target ejection function evaluation value. For example, from six cardiac cycles, the ejection fraction corresponding to the first cardiac cycle, the ejection fraction corresponding to the fourth cardiac cycle, and the ejection fraction corresponding to the sixth cardiac cycle are selected for average processing to obtain a target ejection function evaluation value.

[0103] According to an embodiment of the present invention, the initial volume curve diagram is processed by dividing the cardiac cycle based on the cardiac cycle division rule to obtain a target volume curve diagram containing multiple cardiac cycles, each cardiac cycle only contains one target diastolic volume and one target systolic volume, the ejection fraction is calculated based on the target diastolic volume and the target systolic volume in each cardiac cycle to obtain the ejection fraction corresponding to each cardiac cycle, and then the average value of the multiple ejection fractions is calculated to obtain the target ejection function evaluation value of the left ventricle of the target object, thereby achieving the average ejection fraction value based on multiple cardiac cycles, so as to improve the accuracy of the target ejection function evaluation value and facilitate subsequent operations based on the target ejection function evaluation value.

[0104] According to an embodiment of the present invention, the semi-supervised deep segmentation model is trained in the following manner, and the training method includes the following operations.

[0105] According to an embodiment of the present invention, a training data set is obtained, the training data set includes an ultrasonic cardiology sample video, the ultrasonic cardiology sample video is composed of multiple frames of ultrasonic cardiology sample images, and the multiple frames of ultrasonic cardiology sample images include diastolic sample images containing segmentation labels, contraction sample images containing segmentation labels, and ultrasonic cardiology sample images without segmentation labels.

[0106] According to an embodiment of the present invention, the ultrasound cardiology sample videos in the training data set can be saved using red, green and blue (RGB) three channels, and the ultrasound cardiology sample videos in the training data set can be preprocessed to crop the ultrasound cardiology sample videos to 112×112 pixel sample videos, so that the semi-supervised deep segmentation model to be trained can process and train the sample videos.

[0107] According to an embodiment of the present invention, the ultrasonic cardiology sample video is pre-segmented to obtain a pre-depth segmented image corresponding to each frame of the ultrasonic cardiology sample image in the ultrasonic cardiology sample video.

[0108] According to an embodiment of the present invention, the pre-depth segmentation image includes a pre-depth segmentation label corresponding to each pixel point, and the pre-depth segmentation label represents the depth of each pixel point in the left ventricle region.

[0109] According to an embodiment of the present invention, the diastolic sample image and the systolic sample image containing segmentation labels in the ultrasonic cardiology sample video can be pre-segmented to obtain a pre-depth segmented image corresponding to the diastolic sample image and a pre-depth segmented image corresponding to the systolic sample image.

[0110] According to an embodiment of the present invention, the long axis of the left ventricle in the diastolic sample image and the systolic sample image can be determined first. Based on the long axis of the left ventricle, multiple cross sections perpendicular to the long axis of the left ventricle can be selected, and the width of each cross section is equivalent to the width of the left ventricle. Since the cross sections in the left ventricle of the diastolic sample image and the systolic sample image are not perpendicular to the coordinate system, the diastolic sample image and the systolic sample image can be rotated to facilitate calculation of the depth value corresponding to each pixel point. The rotation angle is shown in formula (2), wherein the relationship between the depth value corresponding to the pixel point and the coordinate of the pixel point in the unrotated coordinate system can be shown according to formula (3).

[0111] (2)

[0112] in, It can be represented as a coordinate system that is rotated perpendicular to the cross section. can be represented as the rotation angle, can be represented as the coordinate system before rotation, u can be represented as the horizontal axis in the coordinate system perpendicular to the cross section after rotation, v can be represented as the vertical axis in the coordinate system perpendicular to the cross section after rotation, x can be represented as the horizontal axis in the coordinate system before rotation, and y can be represented as the vertical axis in the coordinate system before rotation.

[0113] (3)

[0114] Among them, z i It can be represented as the depth value corresponding to the i-th pixel, x i It can be represented as the horizontal coordinate of the i-th pixel, y i It can be represented as the ordinate of the i-th pixel.

[0115] According to an embodiment of the present invention, the left ventricle can be regarded as an ellipsoid, the length R of the cross section in the left ventricle can be regarded as the diameter of the left ventricle disk, the center of each cross section is the center of the disk, and after the diastolic sample image and the systolic sample image are rotated, the radius r from the pixel point to the cross section center can be determined based on the coordinates of the pixel point.

[0116] According to an embodiment of the present invention, when the radius r from the pixel point to the center of the cross-section is greater than the length R of the cross-section, the pixel point is located in the area outside the left ventricle, and the depth segmentation label of the pixel point located in the area outside the left ventricle is pre-set to 0. When the radius r from the pixel point to the center of the cross-section is less than or equal to the length R of the cross-section, the pixel point is located in the area inside the left ventricle. The Pythagorean theorem is used to calculate the depth value corresponding to each pixel point based on the radius r from the pixel point to the center of the cross-section and the length R of the cross-section, that is, the depth segmentation label corresponding to each pixel point.

[0117] According to the embodiment of the present invention, through the above preprocessing, the pre-depth segmentation label corresponding to each pixel point in the diastolic sample image and the systolic sample image can be obtained.

[0118] According to an embodiment of the present invention, the ultrasound cardiology sample video is input into the semi-supervised deep segmentation model to be trained for segmentation processing to obtain a segmentation training image corresponding to each frame of ultrasound sample image in the ultrasound cardiology sample video.

[0119] According to an embodiment of the present invention, the semi-supervised deep segmentation model to be trained is composed of a first image segmentation network and a second image segmentation network with the same initial weights, and the segmentation training images include diastolic training images, contraction training images and multi-frame cardiac training images.

[0120] According to an embodiment of the present invention, the first image segmentation network and the second image segmentation network with the same initial weights can be expressed according to formula (4) and formula (5).

[0121] (4)

[0122] Wherein, P1 can be represented as the first image segmentation network, X can be represented as the ultrasound sample image input to the first image segmentation network, can be characterized as the weights of the first image segmentation network.

[0123] (5)

[0124] Wherein, P2 can be represented as the second image segmentation network, X can be represented as the ultrasound sample image input to the first image segmentation network, It can be represented as the weight of the second image segmentation network.

[0125] According to an embodiment of the present invention, before the semi-supervised deep segmentation model is trained, the initial weights of the two first image segmentation networks and the second image segmentation network may be the same, that is, equal .

[0126] According to an embodiment of the present invention, based on a loss function, a training loss value is obtained according to multiple frames of pre-depth segmentation images, diastolic training images, systolic training images and multiple frames of cardiac training images.

[0127] According to an embodiment of the present invention, a trained semi-supervised deep segmentation model is obtained by adjusting the initial weights of the first image segmentation network and the initial weights of the second image segmentation network according to the training loss value.

[0128] According to an embodiment of the present invention, by acquiring a training data set, the ultrasound cardiology sample video in the training data set is pre-segmented to obtain a pre-depth segmented image corresponding to each frame of ultrasound sample image in the ultrasound cardiology sample video. At the same time, the ultrasound cardiology sample video is input into the semi-supervised deep segmentation model to be trained for segmentation processing to obtain a segmented training image corresponding to each frame of ultrasound sample image in the ultrasound cardiology sample video. Based on the loss function, a training loss value is obtained according to multiple frames of pre-depth segmented images, diastolic training images, contraction training images and multiple frames of cardiac training images. According to the training loss value, the trained semi-supervised deep segmentation model is obtained by adjusting the initial weights of the first image segmentation network and the initial weights of the second image segmentation network. The training of the semi-supervised deep segmentation model is realized, thereby obtaining a semi-supervised deep segmentation model with high robustness. The ultrasound cardiology video is processed using the trained semi-supervised deep segmentation model, and a high-precision target ejection function evaluation value of the left ventricle can be directly output, thereby improving processing efficiency.

[0129] According to an embodiment of the present invention, the diastolic training image includes a first diastolic training image processed by a first image segmentation network and a second diastolic training image processed by a second image segmentation network, the contraction training image includes a first contraction training image processed by the first image segmentation network and a second contraction training image processed by the second image segmentation network, the first diastolic training image and the second diastolic training image include diastolic pixel points, and the first contraction training image and the second contraction training image include contraction pixel points.

[0130] According to an embodiment of the present invention, based on a loss function, a method for obtaining a training loss value according to multiple frames of pre-depth segmentation images, diastolic training images, systolic training images and multiple frames of cardiac training images includes the following operations.

[0131] According to an embodiment of the present invention, for each diastolic pixel in the first diastolic training image, a diastolic loss function of each of the multiple diastolic pixels in the first diastolic training image is obtained according to the training depth of the diastolic pixel and the pre-depth segmentation label of the first target pixel.

[0132] According to an embodiment of the present invention, the first target pixel point is determined from the first pre-depth segmentation image corresponding to the diastolic training image according to the coordinate information of the diastolic pixel point.

[0133] According to an embodiment of the present invention, the multiple frames of ultrasound sample images include two frames of ultrasound sample images containing segmentation labels, namely, a diastolic sample image containing a segmentation label and a systolic sample image containing a segmentation label.

[0134] According to an embodiment of the present invention, a diastolic sample image containing a segmentation label can be first input into a first image segmentation network to obtain a first diastolic training image, a first pre-depth segmentation image corresponding to the diastolic training image can be determined from a plurality of pre-depth segmentation images, and then the coordinate information of the diastolic pixel points can be obtained from the first diastolic training image. According to the coordinate information of the diastolic pixel points, a first target pixel point can be determined from the first pre-depth segmentation image. Finally, according to the training depth of the diastolic pixel point and the pre-depth segmentation label of the first target pixel point, a mean square error calculation is performed to obtain the diastolic loss function of the diastolic pixel point. The above operation is performed on each diastolic pixel point in the first diastolic training image to obtain the diastolic loss function of each of the plurality of diastolic pixel points.

[0135] According to an embodiment of the present invention, for each shrunken pixel in the second shrunken training image, a shrinkage loss function of each of the multiple shrunken pixels in the second shrunken training image is obtained according to the training depth of the shrunken pixel and the pre-depth segmentation label of the second target pixel.

[0136] According to an embodiment of the present invention, the second target pixel point is determined from the second pre-depth segmentation image corresponding to the contraction training image according to the coordinate information of the contraction pixel point.

[0137] According to an embodiment of the present invention, a shrunk sample image containing a segmentation label can be first input into a second image segmentation network to obtain a second shrunk training image, a second pre-depth segmentation image corresponding to the shrunk training image can be determined from multiple pre-depth segmentation images, and then the coordinate information of the shrunk pixel points can be obtained from the second shrunk training image. According to the coordinate information of the shrunk pixel points, the second target pixel points can be determined from the second pre-depth segmentation image. Finally, according to the training depth of the shrunk pixel points and the pre-depth segmentation label of the second target pixel points, the mean square error calculation is performed to obtain the shrinkage loss function of the shrunk pixel points. The above operation is performed on each shrunk pixel point in the second shrunk training image to obtain the shrinkage loss functions of the multiple shrunk pixel points.

[0138] According to an embodiment of the present invention, a training loss value is determined based on a supervision loss value determined based on multiple diastolic loss functions, multiple systolic loss functions, size information of diastolic training images and size information of systolic training images, and a target cross-pseudo-supervisory loss value determined based on size information of multiple frames of cardiac training images, diastolic training images, systolic training images and multiple frames of pre-depth segmentation images.

[0139] According to an embodiment of the present invention, since the diastolic training image corresponds to the diastolic sample image, and the systolic training image corresponds to the systolic sample image, and both the diastolic sample image and the systolic sample image are obtained from the ultrasonic cardiology sample video, the size information of the diastolic training image and the size information of the systolic training image can be the same.

[0140] According to an embodiment of the present invention, the supervision loss value can be calculated according to formula (6).

[0141] (6)

[0142] in, It can be represented as the supervision loss value, can be represented as multiple frames of ultrasound sample images containing segmentation labels, that is, diastolic sample images containing segmentation labels and contraction sample images containing segmentation labels, X can be represented as the Xth ultrasound sample image containing segmentation labels, that is, diastolic sample images containing segmentation labels or contraction sample images containing segmentation labels, W can be represented as the length information of the ultrasound sample image, H can be represented as the width information of the ultrasound sample image, i can be represented as the i-th pixel point, that is, the i-th diastolic pixel point, p li It can be represented as the training depth of the i-th diastolic pixel in the first diastolic training image, p si It can be represented as the training depth of the i-th contracted pixel in the second contracted training image, y * li It can be represented as the pre-depth segmentation label of the first target pixel, y * si It can be represented as the pre-depth segmentation label of the second target pixel, It can be represented as the relaxation loss function of the i-th relaxation pixel in the first relaxation training image, It can be represented as the shrinkage loss function of the i-th shrinkage pixel in the second shrinkage training image.

[0143] According to an embodiment of the present invention, the training loss value can be calculated according to formula (7).

[0144] (7)

[0145] in, It can be represented as the training loss value, It can be represented as the supervision loss value, It can be represented as the target cross pseudo-supervision loss value, It can be characterized as a balance weight.

[0146] According to an embodiment of the present invention, for each diastolic pixel in the first diastolic training image, according to the training depth of the diastolic pixel and the pre-depth segmentation label of the first target pixel, a mean square error is calculated to obtain the diastolic loss function of each of the multiple diastolic pixels in the first diastolic training image; for each contraction pixel in the second contraction training image, according to the training depth of the contraction pixel and the pre-depth segmentation label of the second target pixel, a mean square error is calculated to obtain the contraction loss function of each of the multiple contraction pixels in the second contraction training image; according to the supervised loss value determined based on the multiple diastolic loss functions, the multiple contraction loss functions, the size information of the diastolic training image and the size information of the contraction training image, the training loss value is determined according to the supervised loss value and the target cross pseudo-supervised loss value, thereby implementing the training of the semi-supervised deep segmentation model to be trained, and calculating the loss value using the training result output by the semi-supervised deep segmentation model and the result in the image after pre-depth segmentation, so as to improve the accuracy of the depth segmentation label output by the first image segmentation network and the first image segmentation network for the pixel, and improve the robustness of the semi-supervised deep segmentation model, so as to facilitate the direct application of the trained model to the process of detecting the object to be detected, thereby improving the detection efficiency.

[0147] According to an embodiment of the present invention, the cardiac training image includes a first network training image processed by a first image segmentation network and a second network training image processed by a second image segmentation network, the first network training image includes first cardiac pixel points, and the second network training image includes second cardiac pixel points.

[0148] According to an embodiment of the present invention, the target cross-pseudo-supervision loss value is determined in the following manner, which specifically includes the following operations.

[0149] According to an embodiment of the present invention, a first mean square error function of each of the plurality of first cardiac pixel points in the plurality of first network training images is determined according to the training depth of each first cardiac pixel point and the training depth of the third target pixel point.

[0150] According to an embodiment of the present invention, the third target pixel point is determined from the second network training image based on the coordinate information of the first cardiac pixel point.

[0151] According to an embodiment of the present invention, an ultrasound cardiac sample image without segmentation labels can be simultaneously input into a first image segmentation network and a second image segmentation network to obtain a first network training image and a second network training image, respectively. Then, the coordinate information of the first cardiac pixel point is obtained from the first network training image. According to the coordinate information of the first cardiac pixel point, a third target pixel point is determined from the second network training image. Finally, according to the training depth of the first cardiac pixel point and the training depth of the third target pixel point, a mean square error calculation is performed to obtain a first mean square error function of the first cardiac pixel point. The above operation is performed on each first cardiac pixel point in the first network training image to obtain the first mean square error function of multiple first cardiac pixel points.

[0152] According to an embodiment of the present invention, a second mean square error function of each of the plurality of second cardiac pixel points in the plurality of second network training images is determined according to the training depth of each second cardiac pixel point and the training depth of the fourth target pixel point.

[0153] According to an embodiment of the present invention, the fourth target pixel point is determined from the first network training image based on the coordinate information of the second cardiac pixel point.

[0154] According to an embodiment of the present invention, an ultrasound cardiac sample image without segmentation labels can be simultaneously input into a first image segmentation network and a second image segmentation network to obtain a first network training image and a second network training image, respectively. Then, the coordinate information of the second cardiac pixel point is obtained from the second network training image. According to the coordinate information of the second cardiac pixel point, a fourth target pixel point is determined from the first network training image. Finally, according to the training depth of the second cardiac pixel point and the training depth of the fourth target pixel point, a mean square error calculation is performed to obtain a second mean square error function of the second cardiac pixel point. The above operation is performed on each second cardiac pixel point in the second network training image to obtain the second mean square error function of multiple second cardiac pixel points.

[0155] According to an embodiment of the present invention, each frame of ultrasonic cardiology sample images without segmentation labels may be processed as described above to obtain multiple first mean square error functions and multiple second mean square error functions corresponding to each frame of ultrasonic cardiology sample images.

[0156] According to an embodiment of the present invention, both the diastolic training image and the contraction training image are processed to obtain a third mean square error function of each of the multiple diastolic pixels in the first diastolic training image, a fourth mean square error function of each of the multiple diastolic pixels in the second diastolic training image, a fifth mean square error function of each of the multiple contraction pixels in the first contraction training image, and a sixth mean square error function of each of the multiple contraction pixels in the second contraction training image.

[0157] According to an embodiment of the present invention, the diastolic training image can be simultaneously input into the first image segmentation network and the second image segmentation network to obtain the first diastolic training image and the second diastolic training image respectively, and then the coordinate information of the diastolic pixel point is obtained from the first diastolic training image, and the fifth target pixel point is determined from the second diastolic training image according to the coordinate information of the diastolic pixel point, and the mean square error is calculated according to the training depth of the diastolic pixel point in the first diastolic training image and the training depth of the fifth target pixel point to obtain the third mean square error function of the diastolic pixel point in the first diastolic training image, and for each diastolic pixel point in the first diastolic training image, the third mean square error function of the diastolic pixel point in the first diastolic training image is obtained. The above operation is performed on all pixel points to obtain the third mean square error function of each of the multiple diastolic pixel points; at the same time, the coordinate information of the diastolic pixel points is obtained from the second diastolic training image, and the sixth target pixel point is determined from the first diastolic training image according to the coordinate information of the diastolic pixel point, and the mean square error is calculated according to the training depth of the diastolic pixel points in the second diastolic training image and the training depth of the sixth target pixel point to obtain the fourth mean square error function of the diastolic pixel points in the second diastolic training image, and the above operation is performed on each diastolic pixel point in the second diastolic training image to obtain the fourth mean square error function of each of the multiple diastolic pixel points.

[0158] According to an embodiment of the present invention, the contraction training image can be simultaneously input into the first image segmentation network and the second image segmentation network to obtain the first contraction training image and the second contraction training image respectively, and then the coordinate information of the contraction pixel is obtained from the first contraction training image, and the seventh target pixel is determined from the second contraction training image according to the coordinate information of the contraction pixel. According to the training depth of the contraction pixel in the first contraction training image and the training depth of the seventh target pixel, the mean square error is calculated to obtain the fifth mean square error function of the contraction pixel in the first contraction training image, and the above operation is performed on each contraction pixel in the first contraction training image to obtain the fifth mean square error function of each of the multiple contraction pixels; at the same time, the coordinate information of the contraction pixel is obtained from the second contraction training image, and according to the coordinate information of the contraction pixel, the eighth target pixel is determined from the first contraction training image, and according to the training depth of the contraction pixel in the second contraction training image and the training depth of the eighth target pixel, the mean square error is calculated to obtain the sixth mean square error function of the contraction pixel in the second contraction training image, and the above operation is performed on each contraction pixel in the second contraction training image to obtain the sixth mean square error function of the contraction pixel.

[0159] According to an embodiment of the present invention, a first cross pseudo-supervision loss value is determined based on a plurality of first mean square error functions, a plurality of second mean square error functions and size information of a plurality of frames of cardiac training images.

[0160] According to an embodiment of the present invention, since the ultrasonic cardiology sample images, diastolic sample images and systolic sample images without segmentation labels are all obtained from ultrasonic cardiology sample videos, the size information of the ultrasonic cardiology sample images without segmentation labels and the size information of the diastolic training images, systolic training images and cardiac training images can be the same, and the size information of each frame of the ultrasonic cardiology sample images in the multiple frames of ultrasonic cardiology sample images can be the same.

[0161] According to an embodiment of the present invention, the first cross pseudo-supervision loss value can be calculated according to formula (8).

[0162] (8)

[0163] in, It can be characterized as the first cross pseudo-supervision loss value, can be represented as a multi-frame ultrasound cardiology sample image without segmentation labels, u can be represented as the u-th multi-frame ultrasound cardiology sample image without segmentation labels, W can be represented as the length information of the ultrasound cardiology sample image, H can be represented as the width information of the ultrasound cardiology sample image, i can be represented as the i-th pixel point, that is, the i-th cardiac pixel point, It can be represented as the training depth of the i-th heartbeat pixel in the first network training image, It can be represented as the training depth of the i-th heartbeat pixel in the second network training image, It can be represented as the training depth of the third target pixel corresponding to the i-th cardiac pixel in the second network training image, It can be represented as the training depth of the fourth target pixel corresponding to the i-th cardiac pixel in the first network training image, It can be represented as the first mean square error function of the i-th cardiac pixel, It can be represented as the second mean square error function of the i-th cardiac pixel.

[0164] According to an embodiment of the present invention, a second cross-pseudo-supervised loss value is determined based on multiple third mean square error functions, multiple fourth mean square error functions, multiple fifth mean square error functions, multiple sixth mean square error functions, size information of relaxation training images, and size information of contraction training images.

[0165] According to an embodiment of the present invention, a target cross-pseudo-supervised loss value is determined based on the first cross-pseudo-supervised loss value and the second cross-pseudo-supervised loss value.

[0166] According to an embodiment of the present invention, the target cross-pseudo-supervision loss value can be calculated according to formula (9).

[0167] (9)

[0168] in, It can be represented as the target cross pseudo-supervision loss value, It can be characterized as the second cross pseudo-supervision loss value, It can be characterized as the first cross pseudo-supervision loss value.

[0169] According to an embodiment of the present invention, the second cross-pseudo-supervised loss value is first calculated according to the first cross-pseudo-supervised loss value based on multiple third mean square error functions, multiple fourth mean square error functions and size information of the diastolic training image, to obtain the diastolic cross-pseudo-supervised loss value, and then calculated according to the first cross-pseudo-supervised loss value based on multiple fifth mean square error functions, multiple sixth mean square error functions and size information of the systolic training image, to obtain the systolic cross-pseudo-supervised loss value, and the diastolic cross-pseudo-supervised loss value and the systolic cross-pseudo-supervised loss value are added to obtain the second cross-pseudo-supervised loss value.

[0170] According to an embodiment of the present invention, a first network training image and a second network training image are obtained respectively by inputting a cardiac training image into a first image segmentation network and a second image segmentation network. For the first network training image, a training depth obtained by training the second network training image is used as a reference, and a first mean square error function is constructed with the training depth in the first network training image. For the second network training image, a training depth obtained by training the first network training image is used as a reference, and a second mean square error function is constructed with the training depth in the second network training image. Thus, a first cross-pseudo-supervisory loss value is obtained according to the first mean square error function and the second mean square error function. The above operation is also performed on the diastolic training image and the contraction training image to obtain a second cross-pseudo-supervisory loss value. Supervised loss value, the first cross pseudo-supervised loss value and the second cross pseudo-supervised loss value are added to obtain the target cross pseudo-supervised loss value, and cross-training of the semi-supervised deep segmentation model to be trained is realized using images without segmentation labels to improve the accuracy of the deep segmentation labels output by the semi-supervised deep segmentation model. In addition, while cross-training with sample images without segmentation labels, sample images with segmentation labels are randomly captured and also subjected to the above processing to increase the randomness of the input data, thereby further improving the accuracy of the deep segmentation labels output by the semi-supervised deep segmentation model and the high robustness of the semi-supervised deep segmentation model, so that the trained model can be directly applied to the process of detecting the object under test, thereby improving the detection efficiency.

[0171] Figure 3 A schematic diagram showing the segmentation of the left ventricle in an ultrasound cardiography video according to an embodiment of the present invention is shown.

[0172] like Figure 3 As shown, Figure 3A schematic diagram of the left ventricle after segmentation is shown. The image on the left is an ultrasound image. After magnifying the left ventricle, it can be seen that the left ventricle can be approximated as a cylinder and is segmented into multiple cross sections. The multiple cross sections are parallel to each other, and the axis perpendicular to the multiple cross sections can be the long axis of the left ventricle.

[0173] Figure 4a A schematic diagram showing the regional heat of the left ventricle after segmentation using a semi-supervised deep segmentation model according to an embodiment of the present invention. Figure 4b A schematic diagram showing a regional heat curve of the left ventricle after segmentation using a semi-supervised deep segmentation model according to an embodiment of the present invention is shown.

[0174] like Figure 4a~4b As shown, Figure 4a~4b The regional heat and regional heat curve of the left ventricle after segmentation using the semi-supervised deep segmentation model of the present invention are shown. It can be seen from the figure that the left ventricle can be clearly located through the method of the present invention, so as to calculate the volume inside the left ventricle, thereby obtaining the target ejection function evaluation value of the left ventricle.

[0175] Figure 5 A schematic diagram of a process for obtaining a supervision loss value according to an embodiment of the present invention is shown.

[0176] like Figure 5 As shown, Figure 5 The process of obtaining the supervised loss value of the present invention is shown, X l It can be characterized as a relaxation training image, X s can be characterized as a shrinking training image, It can be characterized as the first image segmentation network, It can be characterized as the second image segmentation network, P l It can be characterized as the first diastolic training image, P s can be characterized as the second contraction training image, P l Output It can be represented as the diastolic loss function, P s Output It can be characterized as a shrinkage loss function, Y1 * It can be characterized as the first pre-depth segmentation image, Y2 * can be characterized as the second pre-depth segmentation image, It can be represented as a supervised loss value. The diastolic training image is input into the first image segmentation network to obtain the first diastolic training image, the contraction training image is input into the second image segmentation network to obtain the second contraction training image, the first pre-depth segmentation image corresponding to the diastolic training image and the second pre-depth segmentation image corresponding to the contraction training image are selected from the multiple frames of pre-depth segmentation images, and the diastolic loss function is constructed according to the pre-depth segmentation label in the first pre-depth segmentation image and the training depth in the first diastolic training image, and the contraction loss function is constructed according to the pre-depth segmentation label in the second pre-depth segmentation image and the training depth in the second contraction training image, and the supervised loss value is obtained according to the contraction loss function and the diastolic loss function.

[0177] Figure 6 A schematic diagram of a process for obtaining a first cross-pseudo-supervision loss value according to an embodiment of the present invention is shown.

[0178] like Figure 6 As shown, Figure 6 The process of obtaining the first cross pseudo-supervision loss value of the present invention is shown, It can be characterized as an ultrasound sample image without segmentation labels. It can be characterized as the first image segmentation network, It can be characterized as a second image segmentation network, P1 can be characterized as a first network training image, P2 can be characterized as a second network training image, Y1 can be characterized as a second network training image for calculating a mean square error function with the first network training image, and Y2 can be characterized as a first network training image for calculating a mean square error function with the second network training image. An ultrasound cardiology sample image without segmentation labels is input into the first image segmentation network and the second image segmentation network to obtain a first network training image and a second network training image respectively, and the second network training image is used as a reference image for calculating a first mean square error function with the first network training image, and the first network training image is used as a reference image for calculating a second mean square error function with the second network training image. A first mean square error function is constructed according to the training depth in the first network training image and the training depth in the second network training image, and a second mean square error function is constructed according to the training depth in the second network training image and the training depth in the first network training image. A first cross pseudo-supervisory loss value is obtained according to the first mean square error function and the second mean square error function.

[0179] Figure 7 A schematic diagram showing the comparison between the ejection fraction obtained by using the model of the present invention and the true value according to an embodiment of the present invention is shown.

[0180] like Figure 7 As shown, Figure 7The figure shows the comparison between the ejection fraction obtained by the method of the present invention and the true value. The horizontal axis can be represented by the true value of the ejection fraction, and the vertical axis can be represented by the measured ejection fraction obtained by the method of the present invention. It can be seen from the figure that the consistency between the measured ejection fraction obtained by the method of the present invention and the true value is high.

[0181] Figure 8 A structural block diagram of a device for evaluating ejection function of a semi-supervised deep segmentation model according to an embodiment of the present invention is shown.

[0182] like Figure 8 As shown, the device for evaluating the ejection function value based on the semi-supervised deep segmentation model of this embodiment includes: an acquisition module 810 , a segmentation module 820 , a calculation module 830 and a obtaining module 840 .

[0183] The acquisition module 810 is used to acquire an ultrasonic cardiology video, wherein the ultrasonic cardiology video includes multiple frames of ultrasonic cardiology images. The acquisition module 810 can be used to perform the operation S210 described above, which will not be described in detail here.

[0184] The segmentation module 820 is used to segment the ultrasound video using a semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel in the left ventricular region in each frame of the ultrasound image, wherein the semi-supervised deep segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the depth of each pixel in the left ventricular region. The segmentation module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0185] The calculation module 830 is used to calculate the candidate volume of the left ventricle in each frame of the ultrasonic cardiogram according to the multiple relative depth estimation values ​​in each frame of the ultrasonic cardiogram. The calculation module 830 can be used to perform the operation S230 described above, which will not be described in detail here.

[0186] The obtaining module 840 is used to determine multiple target systolic volumes and multiple target diastolic volumes from multiple candidate volumes based on a first predetermined rule, and obtain a target ejection function evaluation value of the left ventricle according to the multiple target systolic volumes and the multiple target diastolic volumes, wherein the first predetermined rule indicates that the candidate volume corresponding to the minimum value among the multiple candidate volumes is the target systolic volume, and the candidate volume corresponding to the maximum value among the multiple candidate volumes is the target diastolic volume. The obtaining module 840 can be used to perform the operation S240 described above, which will not be described in detail here.

[0187] According to an embodiment of the present invention, the segmentation module 820 includes: a first obtaining submodule, a second obtaining submodule and a third obtaining submodule.

[0188] The first obtaining submodule is used to input any frame of ultrasound cardiology images in multiple frames of ultrasound cardiology images into a semi-supervised depth segmentation model, so as to calculate the depth value of each pixel point by using the semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point.

[0189] The second obtaining submodule is used to filter out the relative depth estimation value corresponding to each pixel point in the left ventricle area from the multiple relative depth estimation values.

[0190] The third obtaining submodule is used to perform the above operation on each frame of the multiple frames of ultrasonic cardiac images to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricle region in each frame of the ultrasonic cardiac image.

[0191] According to an embodiment of the present invention, the second predetermined rule characterizes that when the relative depth estimation value is 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in an area outside the left ventricle; when the relative depth estimation value is not 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in an area inside the left ventricle.

[0192] According to an embodiment of the present invention, the calculation module 830 includes: a first calculation submodule.

[0193] The first calculation submodule is used to sum the relative depth estimation value of each pixel point in each frame of the ultrasonic cardiogram to calculate the candidate volume of the left ventricle in each frame of the ultrasonic cardiogram.

[0194] According to an embodiment of the present invention, the obtaining module 840 includes: a first generating submodule, a first peak finding submodule and a second calculating submodule.

[0195] The first generating submodule is used to generate an initial volume curve diagram of the left ventricle according to a plurality of candidate volumes.

[0196] The first peak-finding submodule is used to perform peak-finding processing on the initial volume curve graph using a peak-finding function to determine a plurality of target contraction volumes and a plurality of target diastolic volumes.

[0197] The second calculation submodule is used to calculate a target ejection function evaluation value according to a plurality of target systolic volumes and a plurality of target diastolic volumes.

[0198] According to an embodiment of the present invention, the second calculation submodule includes: a first dividing unit, a first calculating unit and a first obtaining unit.

[0199] The first division unit is used to perform cardiac cycle division processing on the initial volume curve graph based on the cardiac cycle division rule to obtain a target volume curve graph, wherein the target volume curve graph represents a curve graph divided into multiple cardiac cycles, and the cardiac cycle division rule represents the division of an adjacent target systolic volume and a target diastolic volume as one cardiac cycle.

[0200] The first calculation unit is used to perform ejection calculation on the target systolic volume and the target diastolic volume in each cardiac cycle to obtain the ejection fraction corresponding to each cardiac cycle.

[0201] The first obtaining unit is used to average multiple ejection fractions to obtain a target ejection function evaluation value.

[0202] According to an embodiment of the present invention, the device for evaluating the ejection function value based on the semi-supervised deep segmentation model further includes: a training module.

[0203] Training module for training semi-supervised deep segmentation models.

[0204] According to an embodiment of the present invention, the training module includes: a first acquisition submodule, a first pre-segmentation submodule, a first segmentation submodule, a fourth acquisition submodule and a first adjustment submodule.

[0205] The first acquisition submodule is used to acquire a training data set, wherein the training data set includes an ultrasonic cardiology sample video, the ultrasonic cardiology sample video is composed of multiple frames of ultrasonic cardiology sample images, and the multiple frames of ultrasonic cardiology sample images include diastolic sample images containing segmentation labels, contraction sample images containing segmentation labels, and ultrasonic cardiology sample images without segmentation labels.

[0206] The first pre-segmentation submodule is used to perform pre-segmentation processing on the ultrasonic cardiology sample video to obtain a pre-depth segmentation image corresponding to each frame of the ultrasonic cardiology sample image in the ultrasonic cardiology sample video, wherein the pre-depth segmentation image includes a pre-depth segmentation label corresponding to each pixel point, and the pre-depth segmentation label represents the depth of each pixel point in the left ventricular area.

[0207] The first segmentation submodule is used to input the ultrasound cardiology sample video into the semi-supervised deep segmentation model to be trained for segmentation processing, so as to obtain a segmentation training image corresponding to each frame of the ultrasound sample image in the ultrasound cardiology sample video, wherein the semi-supervised deep segmentation model to be trained is composed of a first image segmentation network and a second image segmentation network with the same initial weights, and the segmentation training images include a diastolic training image, a contraction training image and a multi-frame cardiac training image.

[0208] The fourth submodule is used to obtain a training loss value based on a loss function according to multiple frames of pre-depth segmentation images, diastolic training images, contraction training images and multiple frames of cardiac training images.

[0209] The first adjustment submodule is used to obtain a trained semi-supervised deep segmentation model by adjusting the initial weights of the first image segmentation network and the initial weights of the second image segmentation network according to the training loss value.

[0210] According to an embodiment of the present invention, the fourth obtaining submodule includes: a second obtaining unit, a third obtaining unit and a first determining unit.

[0211] The second obtaining unit is used to obtain the diastolic loss function of each diastolic pixel point in the first diastolic training image according to the training depth of the diastolic pixel point and the pre-depth segmentation label of the first target pixel point, wherein the first target pixel point is determined from the first pre-depth segmentation image corresponding to the diastolic training image based on the coordinate information of the diastolic pixel point.

[0212] The third obtaining unit is used to obtain, for each shrunken pixel point in the second shrunken training image, a shrinkage loss function of each of the plurality of shrunken pixels in the second shrunken training image according to the training depth of the shrunken pixel point and the pre-depth segmentation label of the second target pixel point, wherein the second target pixel point is determined from the second pre-depth segmentation image corresponding to the shrunken training image according to the coordinate information of the shrunken pixel point.

[0213] The first determination unit is used to determine the training loss value based on the supervision loss value determined based on multiple diastolic loss functions, multiple systolic loss functions, size information of diastolic training images and size information of systolic training images, and the target cross-pseudo-supervised loss value determined based on size information of multiple frames of cardiac training images, diastolic training images, systolic training images and multiple frames of pre-depth segmentation images.

[0214] According to an embodiment of the present invention, the first determining unit includes: a first obtaining subunit, a second obtaining subunit, a third obtaining subunit, a first determining subunit, a second determining subunit and a third determining subunit.

[0215] The first obtaining subunit is used to determine the first mean square error function of each of the multiple first cardiac pixel points in the multiple first network training images according to the training depth of each first cardiac pixel point and the training depth of the third target pixel point, wherein the third target pixel point is determined from the second network training image according to the coordinate information of the first cardiac pixel point.

[0216] The second obtaining subunit is used to determine the second mean square error function of multiple second cardiac pixel points in multiple second network training images according to the training depth of each second cardiac pixel point and the training depth of the fourth target pixel point, wherein the fourth target pixel point is determined from the first network training image according to the coordinate information of the second cardiac pixel point.

[0217] The third obtaining subunit is used to process the diastolic training image and the contraction training image to obtain a third mean square error function of each of the multiple diastolic pixels in the first diastolic training image, a fourth mean square error function of each of the multiple diastolic pixels in the second diastolic training image, a fifth mean square error function of each of the multiple contraction pixels in the first contraction training image, and a sixth mean square error function of each of the multiple contraction pixels in the second contraction training image.

[0218] The first determination subunit is used to determine a first cross pseudo-supervision loss value based on multiple first mean square error functions, multiple second mean square error functions and size information of multiple frames of cardiac training images.

[0219] The second determination subunit is used to determine a second cross pseudo-supervision loss value based on multiple third mean square error functions, multiple fourth mean square error functions, multiple fifth mean square error functions, multiple sixth mean square error functions, size information of the diastolic training image, and size information of the contraction training image.

[0220] The third determination subunit is used to determine a target cross-pseudo-supervision loss value according to the first cross-pseudo-supervision loss value and the second cross-pseudo-supervision loss value.

[0221] According to an embodiment of the present invention, any multiple modules among the acquisition module 810, the segmentation module 820, the calculation module 830 and the acquisition module 840 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 810, the segmentation module 820, the calculation module 830 and the acquisition module 840 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 810, the segmentation module 820, the calculation module 830 and the acquisition module 840 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0222] Fig. 9 A block diagram of an electronic device showing a method for evaluating ejection function based on a semi-supervised deep segmentation model according to an embodiment of the present invention.

[0223] like Fig. 9 As shown, the electronic device according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0224] In RAM 903, various programs and data required for the operation of the electronic device are stored. The processor 901, ROM 902 and RAM 903 are connected to each other via a bus 904. The processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. The processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0225] According to an embodiment of the present invention, the electronic device may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0226] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0227] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0228] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method for evaluating the ejection function value based on the semi-supervised deep segmentation model provided by the embodiment of the present invention.

[0229] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 901. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0230] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0231] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0232] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

Claims

1. A method for evaluating ejection function based on a semi-supervised deep segmentation model, characterized in that: include: Acquiring an ultrasonic cardiology video, wherein the ultrasonic cardiology video includes multiple frames of ultrasonic cardiology images; The ultrasonic video is segmented using a semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel located in the left ventricle region in each frame of the ultrasonic image, wherein the semi-supervised deep segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the depth of each pixel located in the left ventricle region; Calculating a candidate volume of the left ventricle in each frame of the ultrasound cardiogram according to a plurality of relative depth estimation values ​​in each frame of the ultrasound cardiogram; Based on a first predetermined rule, a plurality of target systolic volumes and a plurality of target diastolic volumes are determined from a plurality of candidate volumes, and a target ejection function evaluation value of the left ventricle is obtained according to the plurality of target systolic volumes and the plurality of target diastolic volumes, wherein the first predetermined rule characterizes that a candidate volume corresponding to a minimum value among the plurality of candidate volumes is the target systolic volume, and a candidate volume corresponding to a maximum value among the plurality of candidate volumes is the target diastolic volume.

2. The method according to claim 1, characterized in that The method of using a semi-supervised deep segmentation model to segment the ultrasound video to obtain a relative depth estimation value corresponding to each pixel located in the left ventricle region in each frame of the ultrasound image includes: For any frame of the multiple frames of ultrasound cardiology images, input the any frame of ultrasound cardiology images into the semi-supervised depth segmentation model, so as to calculate the depth value of each pixel point by using the semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point; According to a second predetermined rule, a relative depth estimation value corresponding to each pixel point located in the left ventricle region is obtained by screening from a plurality of relative depth estimation values; The above operation is performed on each frame of the multiple frames of ultrasonic cardiology images to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricle region in each frame of the ultrasonic cardiology images.

3. The method according to claim 2, characterized in that The second predetermined rule represents that when the relative depth estimation value is 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in the area outside the left ventricle; when the relative depth estimation value is not 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in the area inside the left ventricle.

4. The method according to claim 1, characterized in that: The step of calculating the candidate volume of the left ventricle in each frame of the ultrasound cardiogram according to the plurality of relative depth estimation values ​​in each frame of the ultrasound cardiogram comprises: The relative depth estimation value of each pixel in each frame of the ultrasonic cardiogram is summed up to obtain the candidate volume of the left ventricle in each frame of the ultrasonic cardiogram.

5. The method according to claim 1, characterized in that The step of determining a plurality of target systolic volumes and a plurality of target diastolic volumes from a plurality of candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes, comprises: generating an initial volume curve map of the left ventricle according to the plurality of candidate volumes; Performing peak-finding processing on the initial volume curve graph by using a peak-finding function to determine the multiple target contraction volumes and the multiple target diastolic volumes; The target ejection function evaluation value is calculated based on the multiple target systolic volumes and the multiple target diastolic volumes.

6. The method according to claim 5, characterized in that The step of calculating the target ejection function evaluation value according to the multiple target systolic volumes and the multiple target diastolic volumes includes: Based on a cardiac cycle division rule, the initial volume curve graph is processed by cardiac cycle division to obtain a target volume curve graph, wherein the target volume curve graph represents a curve graph including multiple cardiac cycles, and the cardiac cycle division rule represents division of an adjacent target systolic volume and a target diastolic volume as one cardiac cycle; Performing ejection calculation on the target systolic volume and the target diastolic volume in each cardiac cycle to obtain an ejection fraction corresponding to each cardiac cycle; A plurality of ejection fractions are averaged to obtain the target ejection function assessment value.

7. The method according to claim 1, characterized in that The semi-supervised deep segmentation model is trained using the following methods, including: Acquire a training data set, wherein the training data set includes an ultrasound cardiology sample video, the ultrasound cardiology sample video is composed of multiple frames of ultrasound cardiology sample images, and the multiple frames of ultrasound cardiology sample images include diastolic sample images containing segmentation labels, contraction sample images containing segmentation labels, and ultrasound cardiology sample images without segmentation labels; Performing pre-segmentation processing on the ultrasonic cardiology sample video to obtain a pre-depth segmentation image corresponding to each frame of the ultrasonic cardiology sample image in the ultrasonic cardiology sample video, wherein the pre-depth segmentation image includes a pre-depth segmentation label corresponding to each pixel point, and the pre-depth segmentation label represents the depth of each pixel point in the left ventricle region; Inputting the ultrasound cardiology sample video into the semi-supervised deep segmentation model to be trained for segmentation processing, and obtaining a segmentation training image corresponding to each frame of ultrasound sample image in the ultrasound cardiology sample video, wherein the semi-supervised deep segmentation model to be trained is composed of a first image segmentation network and a second image segmentation network with the same initial weights, and the segmentation training images include a diastolic training image, a systolic training image, and multiple frames of cardiac training images; Based on the loss function, a training loss value is obtained according to the multiple frames of the pre-depth segmentation images, the diastolic training images, the systolic training images and the multiple frames of cardiac training images; According to the training loss value, the trained semi-supervised deep segmentation model is obtained by adjusting the initial weights of the first image segmentation network and the initial weights of the second image segmentation network.

8. The method according to claim 7, characterized in that The diastolic training image includes a first diastolic training image processed by the first image segmentation network and a second diastolic training image processed by the second image segmentation network, the contraction training image includes a first contraction training image processed by the first image segmentation network and a second contraction training image processed by the second image segmentation network, the first diastolic training image and the second diastolic training image include diastolic pixels, and the first contraction training image and the second contraction training image include contraction pixels; The method of obtaining a training loss value based on the loss function according to the multiple frames of the pre-depth segmentation images, the diastolic training images, the systolic training images and the multiple frames of cardiac training images comprises: For each of the diastolic pixel points in the first diastolic training image, obtaining respective diastolic loss functions of the plurality of diastolic pixel points in the first diastolic training image according to the training depth of the diastolic pixel point and the pre-depth segmentation label of the first target pixel point, wherein the first target pixel point is determined from the first pre-depth segmentation image corresponding to the diastolic training image according to the coordinate information of the diastolic pixel point; For each of the shrunken pixel points in the second shrunken training image, obtaining a respective shrunken loss function of a plurality of the shrunken pixel points in the second shrunken training image according to the training depth of the shrunken pixel point and the pre-depth segmentation label of the second target pixel point, wherein the second target pixel point is determined from a second pre-depth segmentation image corresponding to the shrunken training image according to the coordinate information of the shrunken pixel point; The training loss value is determined according to the supervised loss value determined based on the multiple diastolic loss functions, the multiple systolic loss functions, the size information of the diastolic training image and the size information of the systolic training image, and the target cross-pseudo-supervised loss value determined based on the size information of the multiple frames of cardiac training images, the diastolic training images, the systolic training images and the multiple frames of pre-depth segmentation images.

9. The method according to claim 8, characterized in that The heartbeat training image includes a first network training image processed by the first image segmentation network and a second network training image processed by the second image segmentation network, the first network training image includes first heartbeat pixels, and the second network training image includes second heartbeat pixels; The target cross pseudo-supervision loss value is determined as follows: Determine, according to the training depth of each of the first cardiac pixel points and the training depth of the third target pixel point, a first mean square error function of each of the first cardiac pixel points in the first network training images, wherein the third target pixel point is determined from the second network training image according to the coordinate information of the first cardiac pixel point; Determine, according to the training depth of each of the second cardiac pixel points and the training depth of the fourth target pixel point, a second mean square error function of each of the second cardiac pixel points in the second network training images, wherein the fourth target pixel point is determined from the first network training image according to the coordinate information of the second cardiac pixel point; Processing the diastolic training image and the systolic training image to obtain a third mean square error function of each of the diastolic pixels in the first diastolic training image, a fourth mean square error function of each of the diastolic pixels in the second diastolic training image, a fifth mean square error function of each of the systolic pixels in the first systolic training image, and a sixth mean square error function of each of the systolic pixels in the second systolic training image; Determine a first cross pseudo-supervision loss value based on the multiple first mean square error functions, the multiple second mean square error functions and size information of the multiple frames of cardiac training images; Determining a second cross pseudo-supervision loss value based on the plurality of third mean square error functions, the plurality of fourth mean square error functions, the plurality of fifth mean square error functions, the plurality of sixth mean square error functions, the size information of the relaxation training image, and the size information of the contraction training image; A target cross-pseudo-supervised loss value is determined according to the first cross-pseudo-supervised loss value and the second cross-pseudo-supervised loss value.

10. A device for evaluating ejection function based on a semi-supervised deep segmentation model, characterized in that: include: An acquisition module, used for acquiring an ultrasonic cardiology video, wherein the ultrasonic cardiology video includes multiple frames of ultrasonic cardiology images; A segmentation module, configured to perform segmentation processing on the ultrasound video using a semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel located in the left ventricle region in each frame of the ultrasound image, wherein the semi-supervised deep segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the depth of each pixel located in the left ventricle region; A calculation module, configured to calculate a candidate volume of the left ventricle in each frame of the ultrasonic cardiogram according to a plurality of relative depth estimation values ​​in each frame of the ultrasonic cardiogram; an obtaining module, for determining a plurality of target systolic volumes and a plurality of target diastolic volumes from a plurality of candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes, wherein the first predetermined rule indicates that the candidate volume corresponding to the minimum value among the plurality of candidate volumes is the target systolic volume, and the candidate volume corresponding to the maximum value among the plurality of candidate volumes is the target diastolic volume.

Images