Method and device for evaluating ejection function value based on semi-supervised depth segmentation model

Abstract

The present invention provides a method and device for evaluating the ejection function value based on a semi-supervised deep segmentation model, which can be applied to the field of medical detection technology. The method for evaluating the ejection function value based on the semi-supervised deep segmentation model includes: obtaining an echocardiogram video; using the semi-supervised deep segmentation model to perform segmentation processing on the echocardiogram video to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of echocardiogram image; calculating a candidate volume of the left ventricle in each frame of echocardiogram image according to a plurality of relative depth estimation values in each frame of echocardiogram image; determining a plurality of target systolic volumes and a plurality of target diastolic volumes from the plurality of candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes.

Description

Technical Field

[0001] The present invention relates to the technical field of medical detection, and more particularly to a method and device for evaluating the ejection function value based on a semi-supervised deep segmentation model. Background Art

[0002] Since the echocardiogram of the obtained object has real-time performance, and the process of obtaining the echocardiogram of the object by an ultrasonic device has advantages such as low cost and no radiation, it is a common detection method at present to use the non-invasive echocardiogram as an intermediate result to perform targeted diagnosis on the above object. Further, the left ventricular ejection fraction is also one of the important data for helping to diagnose the pumping function of the above object. However, there is a large error between the left ventricular ejection fraction predicted according to the existing echocardiogram or echocardiogram video and the real result.

[0003] In the process of implementing the above inventive concept, it is found that: in the related art, due to the inability to accurately process the echocardiogram video to obtain the end-diastolic frame and end-systolic frame in the echocardiogram video, it is impossible to obtain an accurate left ventricular ejection fraction, and at the same time, when processing the end-diastolic frame and end-systolic frame to estimate the left ventricular ejection fraction, it is impossible to intuitively see the intermediate estimation process and intermediate estimation results, so that the estimation process cannot be regulated, resulting in the technical problems of large error and poor flexibility of the left ventricular ejection fraction. Summary of the Invention

[0004] In view of the above problems, the present invention provides a method and device for evaluating the ejection function value based on a semi-supervised deep segmentation model.

[0005] According to a first aspect of the present invention, there is provided a method for evaluating the ejection function value based on a semi-supervised deep segmentation model, including: obtaining an echocardiogram video, where the echocardiogram video includes multiple frames of echocardiogram images; using the semi-supervised deep segmentation model to perform segmentation processing on the echocardiogram video to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of echocardiogram image, where the semi-supervised deep segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the depth of each pixel point in the left ventricular region; calculating a candidate volume of the left ventricle in each frame of echocardiogram image according to the multiple relative depth estimation values in each frame of echocardiogram image; determining multiple target systolic volumes and multiple target diastolic volumes from the multiple candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the multiple target systolic volumes and multiple target diastolic volumes, where the first predetermined rule represents that the candidate volume corresponding to the minimum value among the multiple candidate volumes is the target systolic volume, and the candidate volume corresponding to the maximum value among the multiple candidate volumes is the target diastolic volume.

[0006] According to an embodiment of the present invention, a semi-supervised depth segmentation model is used to segment an echocardiogram video, and relative depth estimation values corresponding to each pixel point located in the left ventricular region are obtained in each frame of echocardiogram image, including: for any one frame of echocardiogram images among multiple frames of echocardiogram images, inputting any one frame of echocardiogram image into the semi-supervised depth segmentation model to calculate the depth value of each pixel point by using the semi-supervised depth segmentation model, and obtaining relative depth estimation values corresponding to each pixel point; according to a second predetermined rule, screening out relative depth estimation values corresponding to each pixel point located in the left ventricular region from multiple relative depth estimation values; performing the above operations on each frame of echocardiogram images among multiple frames of echocardiogram images to obtain relative depth estimation values corresponding to each pixel point located in the left ventricular region in each frame of echocardiogram image.

[0007] According to an embodiment of the present invention, the second predetermined rule indicates that when the relative depth estimation value is 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in a region outside the left ventricle, and when the relative depth estimation value is not 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in a region inside the left ventricle.

[0008] According to an embodiment of the present invention, based on multiple relative depth estimation values in each frame of echocardiogram image, a candidate volume of the left ventricle is calculated in each frame of echocardiogram image, including: performing a summation process on the relative depth estimation values of each pixel point in each frame of echocardiogram image to calculate the candidate volume of the left ventricle in each frame of echocardiogram image.

[0009] According to an embodiment of the present invention, based on a first predetermined rule, multiple target systolic volumes and multiple target diastolic volumes are determined from multiple candidate volumes, and based on the multiple target systolic volumes and multiple target diastolic volumes, a target ejection function evaluation value of the left ventricle is obtained, including: generating an initial volume curve graph of the left ventricle according to the multiple candidate volumes; using a peak seeking function to perform peak seeking processing on the initial volume curve graph to determine multiple target systolic volumes and multiple target diastolic volumes; calculating the target ejection function evaluation value according to the multiple target systolic volumes and multiple target diastolic volumes.

[0010] According to an embodiment of the present invention, a target ejection function evaluation value is calculated based on a plurality of target systolic volumes and a plurality of target diastolic volumes, including: performing a cardiac cycle division process on an initial volume curve graph based on a cardiac cycle division rule to obtain a target volume curve graph, where the target volume curve graph represents a graph divided into a plurality of cardiac cycles, and the cardiac cycle division rule represents dividing with an adjacent target systolic volume and a target diastolic volume as one cardiac cycle; performing an ejection calculation on the target systolic volume and the target diastolic volume within each cardiac cycle to obtain an ejection fraction corresponding to each cardiac cycle; and performing an averaging process on the plurality of ejection fractions to obtain the target ejection function evaluation value.

[0011] According to an embodiment of the present invention, the semi-supervised deep segmentation model is trained by the following method, including: obtaining a training data set, where the training data set includes an echocardiogram sample video, and the echocardiogram sample video is composed of multiple frames of echocardiogram sample images. Among the multiple frames of echocardiogram sample images, there are diastolic sample images with segmentation labels, systolic sample images with segmentation labels, and echocardiogram sample images without segmentation labels; performing a pre-segmentation process on the echocardiogram sample video to obtain a pre-depth segmentation image corresponding to each frame of echocardiogram sample image in the echocardiogram sample video, where the pre-depth segmentation image includes a pre-depth segmentation label corresponding to each pixel point, and the pre-depth segmentation label represents the depth of each pixel point in the left ventricular region; inputting the echocardiogram sample video into the semi-supervised deep segmentation model to be trained for segmentation processing to obtain a segmentation training image corresponding to each frame of echocardiogram sample image in the echocardiogram sample video, where the semi-supervised deep segmentation model to be trained is composed of two first image segmentation networks and a second image segmentation network with the same initial weights, and the segmentation training images include diastolic training images, systolic training images, and multiple frames of cardiac cycle training images; based on a loss function, obtaining a training loss value according to the multiple frames of pre-depth segmentation images, diastolic training images, systolic training images, and multiple frames of cardiac cycle training images; and obtaining the trained semi-supervised deep segmentation model by adjusting the initial weights of the first image segmentation network and the second image segmentation network according to the training loss value.

[0012] According to an embodiment of the present invention, the diastolic training images include a first diastolic training image processed by a first image segmentation network and a second diastolic training image processed by a second image segmentation network, the systolic training images include a first systolic training image processed by the first image segmentation network and a second systolic training image processed by the second image segmentation network, the first diastolic training image and the second diastolic training image include diastolic pixel points, and the first systolic training image and the second systolic training image include systolic pixel points; based on a loss function, a training loss value is obtained according to multiple pre-depth segmentation images, diastolic training images, systolic training images, and multiple cardiac training images, including: for each diastolic pixel point in the first diastolic training image, a diastolic loss function for each of the multiple diastolic pixel points in the first diastolic training image is obtained according to the training depth of the diastolic pixel point and the pre-depth segmentation label of the first target pixel point, where the first target pixel point is determined from the first pre-depth segmentation image corresponding to the diastolic training image according to the coordinate information of the diastolic pixel point; for each systolic pixel point in the second systolic training image, a systolic loss function for each of the multiple systolic pixel points in the second systolic training image is obtained according to the training depth of the systolic pixel point and the pre-depth segmentation label of the second target pixel point, where the second target pixel point is determined from the second pre-depth segmentation image corresponding to the systolic training image according to the coordinate information of the systolic pixel point; the training loss value is determined according to a supervised loss value determined based on multiple diastolic loss functions, multiple systolic loss functions, the size information of the diastolic training image, and the size information of the systolic training image, and a target cross pseudo-supervised loss value determined based on the size information of the multiple cardiac training images, the diastolic training image, the systolic training image, and the multiple pre-depth segmentation images.

[0013] According to an embodiment of the present invention, the cardiac training images include a first network training image processed by a first image segmentation network and a second network training image processed by a second image segmentation network. The first network training image includes first cardiac pixels, and the second network training image includes second cardiac pixels; the target cross pseudo-supervision loss value is determined in the following manner: according to the training depth of each first cardiac pixel and the training depth of a third target pixel, determine the respective first mean square error functions of the multiple first cardiac pixels in the multiple first network training images, wherein the third target pixel is determined from the second network training image according to the coordinate information of the first cardiac pixel; according to the training depth of each second cardiac pixel and the training depth of a fourth target pixel, determine the respective second mean square error functions of the multiple second cardiac pixels in the multiple second network training images, wherein the fourth target pixel is determined from the first network training image according to the coordinate information of the second cardiac pixel; process the diastolic training image and the systolic training image to obtain the respective third mean square error functions of the multiple diastolic pixels in the first diastolic training image, the respective fourth mean square error functions of the multiple diastolic pixels in the second diastolic training image, the respective fifth mean square error functions of the multiple systolic pixels in the first systolic training image, and the respective sixth mean square error functions of the multiple systolic pixels in the second systolic training image; based on the multiple first mean square error functions, the multiple second mean square error functions, and the size information of the multiple frames of cardiac training images, determine a first cross pseudo-supervision loss value, and based on the multiple third mean square error functions, the multiple fourth mean square error functions, the multiple fifth mean square error functions, the multiple sixth mean square error functions, the size information of the diastolic training image, and the size information of the systolic training image, determine a second cross pseudo-supervision loss value, and according to the first cross pseudo-supervision loss value and the second cross pseudo-supervision loss value, determine the target cross pseudo-supervision loss value.

[0014] The second aspect of the present invention provides a device for evaluating the ejection function value based on a semi-supervised depth segmentation model, which is characterized by including: an acquisition module for acquiring an echocardiogram video, where the echocardiogram video includes multiple frames of echocardiogram images; a segmentation module for performing segmentation processing on the echocardiogram video by using the semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of echocardiogram image, where the semi-supervised depth segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the depth of each pixel point in the left ventricular region; a calculation module for calculating the candidate volume of the left ventricle in each frame of echocardiogram image according to multiple relative depth estimation values in each frame of echocardiogram image; an obtaining module for determining multiple target systolic volumes and multiple target diastolic volumes from multiple candidate volumes based on a first predetermined rule, and obtaining the target ejection function value of the left ventricle according to the multiple target systolic volumes and multiple target diastolic volumes, where the first predetermined rule represents that the candidate volume corresponding to the minimum value among the multiple candidate volumes is the target systolic volume, and the candidate volume corresponding to the maximum value among the multiple candidate volumes is the target diastolic volume.

[0015] The third aspect of the present invention provides an electronic device, including: one or more processors; a memory for storing one or more programs, where when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the above method.

[0016] The fourth aspect of the present invention further provides a computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor is caused to execute the above method.

[0017] The fifth aspect of the present invention further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0018] The method and device for evaluating the ejection function value based on a semi-supervised depth segmentation model according to the present invention obtain an echocardiogram video, use the semi-supervised depth segmentation model to perform segmentation processing on the echocardiogram video, obtain the relative depth estimation value corresponding to each pixel point located in the left ventricle region in each frame of echocardiogram image, calculate the candidate volume of the left ventricle in each frame of echocardiogram image according to the multiple relative depth estimation values in each frame of echocardiogram image, determine multiple target systolic volumes and multiple target diastolic volumes from the multiple candidate volumes based on a first predetermined rule, and obtain the target ejection function evaluation value of the left ventricle according to the multiple target systolic volumes and multiple target diastolic volumes. It realizes obtaining the accurate target ejection function evaluation value of the left ventricle of the target object according to the echocardiogram video by using a pre-trained model, and determines the target diastolic volume and target systolic volume based on the maximum value and minimum value in the candidate volume, so as to accurately locate the end-diastolic frame and end-systolic frame in the echocardiogram video, facilitating the obtaining of a high-accuracy and high-robustness left ventricular ejection fraction. And in the process of obtaining the target ejection function evaluation value of the left ventricle of the target object, relevant professionals can call the relative depth estimation value of each pixel point in any frame of echocardiogram image, multiple candidate volumes of the left ventricle, end-diastolic frame image, end-systolic frame image and other parameters of the semi-supervised depth segmentation model at any time during the operation of the model, so as to facilitate the relevant professionals to regulate the semi-supervised depth segmentation model, make the intermediate process visual, and improve the convenience and interactivity of the operation.

[0019] According to an embodiment of the present invention, further, since the semi-supervised depth segmentation model is built according to the first image segmentation network and the second image segmentation network, and then a large number of model trainings are performed on the built semi-supervised depth segmentation model to obtain a trained semi-supervised depth segmentation model. In the case of estimating the left ventricular ejection fraction of the target object, the semi-supervised depth segmentation model is directly used to process the echocardiogram video to obtain the target ejection function evaluation value of the left ventricle. It realizes using a semi-supervised depth segmentation model with high robustness to process the echocardiogram video, improves work efficiency, and reduces work costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Through the following description of the embodiments of the present invention with reference to the drawings, the above content and other objects, features and advantages of the present invention will become clearer. In the drawings:

[0021] Figure 1 An application scenario diagram of the method for evaluating the ejection function value based on a semi-supervised depth segmentation model according to an embodiment of the present invention is shown;

[0022] Figure 2 A flowchart of the method for evaluating the ejection function value based on a semi-supervised depth segmentation model according to an embodiment of the present invention is shown;

[0023] Figure 3 Shows a schematic diagram of the left ventricle segmentation labels in an echocardiogram video according to an embodiment of the present invention;

[0024] Figure 4a Shows a schematic diagram of the regional heat map of the left ventricle after segmentation using a semi-supervised deep segmentation model according to an embodiment of the present invention;

[0025] Figure 4b Shows a schematic diagram of the regional heat curve of the left ventricle after segmentation using a semi-supervised deep segmentation model according to an embodiment of the present invention;

[0026] Figure 5 Shows a schematic diagram of the process of obtaining a supervised loss value according to an embodiment of the present invention;

[0027] Figure 6 Shows a schematic diagram of the process of obtaining a cross-pseudo-supervised loss value according to an embodiment of the present invention;

[0028] Figure 7 Shows a schematic diagram of the comparison between the ejection fraction obtained using the model of the present invention and the true value according to an embodiment of the present invention;

[0029] Figure 8 Shows a structural block diagram of a device for evaluating the ejection function value of a semi-supervised deep segmentation model according to an embodiment of the present invention;

[0030] Figure 9 Shows a block diagram of an electronic device for a method of evaluating the ejection function value of a semi-supervised deep segmentation model according to an embodiment of the present invention. Detailed implementation manners

[0031] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, many specific details are set forth in order to provide a comprehensive understanding of the embodiments of the present invention. However, it is obvious that one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present invention.

[0032] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0033] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0034] In cases where expressions similar to "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0035] In the technical solution of the present invention, the user information involved (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties. Moreover, the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, all comply with relevant laws, regulations, and standards, adopt necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or reject.

[0036] Echocardiography is a non-invasive bioimaging examination method that uses ultrasonic technology to observe the structure of a target, such as the structure of the heart, in order to assist in targeted diagnosis of the target as an intermediate result. Due to the advantages of high real-time performance, low cost, and no radiation of echocardiography, it is widely used in the evaluation of the operating function of the target. However, limited by problems such as the noise and contrast of echocardiography, it is impossible to obtain accurate left ventricular volume, resulting in the inability to assist in targeted diagnosis of the target. At the same time, the ventricular ejection fraction is one of the important auxiliary judgment indicators of cardiac pumping function in clinical reports. Moreover, the process of estimating the left ventricular volume based on echocardiography is very time-consuming. Therefore, the prior art processes echocardiography videos to obtain the left ventricular ejection fraction.

[0037] The existing measurement of left ventricular ejection fraction usually finds the end-diastolic frame and end-systolic frame of the left ventricle by independently segmenting the left ventricular region of each frame, estimates the end-diastolic volume and end-systolic volume according to the Simpson's method, and thus estimates the left ventricular ejection fraction. However, the existing method for obtaining the left ventricular ejection fraction from echocardiographic videos can neither control the specific operation nor obtain the situation of the target during the prediction process, so it is impossible to judge the accuracy of the obtained left ventricular ejection fraction and it cannot be widely applied to actual operations. During the R & D process, the R & D personnel found that in the related technologies, due to the inability to accurately obtain the end-diastolic frame and end-systolic frame in the echocardiographic video, it is impossible to obtain an accurate left ventricular ejection fraction. Moreover, during the estimation of the left ventricular ejection fraction, it is impossible to intuitively see the intermediate estimation process and intermediate estimation results, so it is impossible to control the estimation process, resulting in technical problems of large error and poor flexibility of the left ventricular ejection fraction.

[0038] In view of this, an embodiment of the present invention provides a method for evaluating the ejection function value based on a semi-supervised deep segmentation model, including: obtaining an echocardiographic video, where the echocardiographic video includes multiple frames of echocardiographic images; using the semi-supervised deep segmentation model to perform segmentation processing on the echocardiographic video to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of echocardiographic image, where the semi-supervised deep segmentation model includes a first image segmentation convolutional neural network and a second image segmentation convolutional neural network with different weights, and the relative depth estimation value represents the depth of each pixel point in the left ventricular region; calculating the candidate volume of the left ventricle in each frame of echocardiographic image according to the multiple relative depth estimation values in each frame of echocardiographic image; determining multiple target systolic volumes and multiple target diastolic volumes from the multiple candidate volumes based on a first predetermined rule, and obtaining the target ejection function value of the left ventricle according to the multiple target systolic volumes and multiple target diastolic volumes, where the first predetermined rule represents that the candidate volume corresponding to the minimum value among the multiple candidate volumes is the target systolic volume, and the candidate volume corresponding to the maximum value among the multiple candidate volumes is the target diastolic volume.

[0039] Figure 1 The application scenario diagram of the method for evaluating the ejection function value based on the semi-supervised deep segmentation model according to the embodiment of the present invention is shown.

[0040] As Figure 1 shown, the application scenario according to this embodiment may include a first ultrasonic device 101, a target object 102, and a receiver 103. The first ultrasonic device 101 is used to send ultrasonic signals to the target object 102.

[0041] The user can use the first ultrasonic device 101 to interact with the target object 102 and the receiver 103 to receive or send signals, etc.

[0042] The receiver 103 can be a receiver that receives various ultrasonic signals. For example, it receives and processes the ultrasonic signals sent by the first ultrasonic device 101 (only for illustration). The receiver 103 can analyze and process data such as the received ultrasonic signals, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0043] It should be noted that the method for evaluating the ejection function value based on the semi-supervised deep segmentation model provided by the embodiments of the present invention can generally be executed by the receiver 103. Correspondingly, the device for evaluating the ejection function value based on the semi-supervised deep segmentation model provided by the embodiments of the present invention can generally be set in the receiver 103. The method for evaluating the ejection function value based on the semi-supervised deep segmentation model provided by the embodiments of the present invention can also be executed by a receiver or a cluster of receivers that is different from the receiver 103 and can communicate with the first ultrasonic device 101 and / or the receiver 103. Correspondingly, the device for evaluating the ejection function value based on the semi-supervised deep segmentation model provided by the embodiments of the present invention can also be set in a receiver or a cluster of receivers that is different from the receiver 103 and can communicate with the first ultrasonic device 101 and / or the receiver 103.

[0044] It should be understood that Figure 1 the numbers of the first ultrasonic device, the target object, and the receiver in are merely illustrative. According to the implementation requirements, any number of ultrasonic devices, observation objects, and receivers can be provided.

[0045] Based on the Figure 1 scenario described below, the method for evaluating the ejection function value based on the semi-supervised deep segmentation model of the embodiments of the invention will be described in detail through Figures 2 to 7

[0046] Figure 2 FIG. shows a flowchart of the method for evaluating the ejection function value based on the semi-supervised deep segmentation model according to an embodiment of the present invention.

[0047] As Figure 2 shown, the method for evaluating the ejection function value based on the semi-supervised deep segmentation model of this embodiment includes operations S210 to S240.

[0048] In operation S210, an echocardiogram video is acquired.

[0049] According to an embodiment of the present invention, the echocardiogram video includes multiple frames of echocardiogram images.

[0050] According to an embodiment of the present invention, each frame of echocardiogram image of the obtained target includes the region of the apical four-chamber view, and the apical four-chamber view includes the left ventricular region, the left atrial region, the right ventricular region, and the right atrial region.

[0051] According to an embodiment of the present invention, each frame of echocardiogram image in the echocardiogram video can be separately extracted for review.

[0052] In operation S220, the echocardiogram video is segmented by using a semi-supervised depth segmentation model to obtain the relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of echocardiogram image.

[0053] According to an embodiment of the present invention, the semi-supervised depth segmentation model includes a first image segmentation network and a second image segmentation network with different weights.

[0054] According to an embodiment of the present invention, the relative depth estimation value represents the depth of each pixel point in the left ventricular region.

[0055] According to an embodiment of the present invention, the untrained semi-supervised depth segmentation model is pre-trained to obtain a trained semi-supervised depth segmentation model. The trained semi-supervised depth segmentation model is used to calculate the segmentation depth of each frame of echocardiogram image in the obtained echocardiogram video, and the relative depth estimation value corresponding to each pixel point in each frame of echocardiogram image is obtained. Whether the pixel point is located in the left ventricular region can be judged according to the depth value corresponding to each pixel point, so as to obtain the relative depth estimation value corresponding to each pixel point located in the left ventricular region.

[0056] According to an embodiment of the present invention, each frame of echocardiogram image containing the relative depth estimation value corresponding to each pixel point can be separately extracted for review, so as to facilitate relevant professionals to view the depth value of any pixel point, thereby adjusting the model.

[0057] In operation S230, according to the multiple relative depth estimation values in each frame of echocardiogram image, the candidate volume of the left ventricle in each frame of echocardiogram image is calculated.

[0058] According to an embodiment of the present invention, by calculating the candidate volume of the left ventricle in each frame of echocardiogram image, it is convenient to calculate the left ventricular ejection fraction of the target, that is, the target ejection function evaluation value.

[0059] According to an embodiment of the present invention, the candidate volume of the left ventricle in each frame of echocardiogram image can be separately extracted for review.

[0060] In operation S240, based on a first predetermined rule, a plurality of target systolic volumes and a plurality of target diastolic volumes are determined from a plurality of candidate volumes, and based on the plurality of target systolic volumes and the plurality of target diastolic volumes, a target ejection function evaluation value of the left ventricle is obtained.

[0061] According to an embodiment of the present invention, the first predetermined rule characterizes that the candidate volume corresponding to the minimum value among the plurality of candidate volumes is the target systolic volume, and the candidate volume corresponding to the maximum value among the plurality of candidate volumes is the target diastolic volume.

[0062] According to an embodiment of the present invention, the target ejection function evaluation value of the left ventricle can be characterized as the target ejection fraction of the left ventricle.

[0063] According to an embodiment of the present invention, a volume curve graph corresponding to the left ventricle can be generated based on the plurality of candidate volumes, a plurality of minimum value points and maximum value points are determined from the curve graph based on the first predetermined rule, the plurality of minimum value points are determined as the plurality of target systolic volumes, and the plurality of echocardiogram images corresponding to the plurality of minimum value points can be determined as the target end-systolic frames, the plurality of maximum value points are determined as the plurality of target diastolic volumes, and the plurality of echocardiogram images corresponding to the plurality of maximum value points can be determined as the target end-diastolic frames.

[0064] According to an embodiment of the present invention, based on the obtained target ejection function evaluation value of the left ventricle, it can help relevant personnel to judge the relevant functions of the target.

[0065] For example, an ultrasonic examination is performed on the heart of a target object to obtain an echocardiogram video. The echocardiogram video is segmented using a trained semi-supervised deep segmentation model to obtain the depth value corresponding to each pixel point in the left ventricle region of each frame of echocardiogram image, that is, the relative depth estimation value. The candidate volume of the left ventricle in each frame of echocardiogram image is calculated based on the relative depth estimation value of each pixel point in each frame of echocardiogram image. A plurality of target systolic volumes and a plurality of target diastolic volumes are selected from the plurality of candidate volumes, and the target ejection function evaluation value of the left ventricle in the heart of the target object is calculated based on the plurality of target systolic volumes and the plurality of target diastolic volumes, so that scientific personnel or professionals in related fields can distinguish the cardiac function of the target object according to the target ejection function evaluation value of the left ventricle, and thus make a judgment suitable for the target object.

[0066] According to an embodiment of the present invention, each target diastolic volume, target systolic volume, end-systolic frame image, and end-diastolic frame image can be separately extracted for review.

[0067] According to an embodiment of the present invention, by acquiring an echocardiogram video, using a semi-supervised depth segmentation model to segment the echocardiogram video, obtaining relative depth estimation values corresponding to each pixel point located in the left ventricular region in each frame of the echocardiogram image, calculating candidate volumes of the left ventricle in each frame of the echocardiogram image based on the multiple relative depth estimation values in each frame of the echocardiogram image, determining multiple target systolic volumes and multiple target diastolic volumes from the multiple candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the multiple target systolic volumes and the multiple target diastolic volumes. It realizes obtaining an accurate target ejection function evaluation value of the left ventricle of the target object according to the echocardiogram video by using a pre-trained model, and determining the target diastolic volume and the target systolic volume based on the maximum value and the minimum value in the candidate volume, so as to accurately locate the end-diastolic frame and the end-systolic frame in the echocardiogram video, so as to obtain a high-accuracy and high-robustness left ventricular ejection fraction. And in the process of obtaining the target ejection function evaluation value of the left ventricle of the target object, relevant professionals can call the relative depth estimation value of each pixel point in any frame of the echocardiogram image, multiple candidate volumes of the left ventricle, end-diastolic frame images and end-systolic frame images and other parameters of the semi-supervised depth segmentation model at any time during the working process of the model, so as to facilitate relevant professionals to adjust the semi-supervised depth segmentation model, make the intermediate process visual, and improve the convenience and interactivity of the operation.

[0068] According to an embodiment of the present invention, further, since a semi-supervised depth segmentation model is built based on the first image segmentation network and the second image segmentation network, and then a large number of model trainings are performed on the built semi-supervised depth segmentation model to obtain a trained semi-supervised depth segmentation model. In the case of estimating the left ventricular ejection fraction of the target object, directly use the semi-supervised depth segmentation model to process the echocardiogram video to obtain a target ejection function evaluation value of the left ventricle. It realizes using a semi-supervised depth segmentation model with high robustness to process the echocardiogram video, improves work efficiency, and reduces work costs.

[0069] It should be noted that the target ejection function evaluation value of the left ventricle obtained by the present invention is only used as an intermediate result, and a diagnostic result or health status cannot be directly obtained from the target ejection function evaluation value of the left ventricle obtained by the method according to the present invention.

[0070] According to an embodiment of the present invention, the method for using a semi-supervised depth segmentation model to segment an echocardiogram video to obtain relative depth estimation values corresponding to each pixel point located in the left ventricular region in each frame of the echocardiogram image includes the following operations.

[0071] According to an embodiment of the present invention, for any one of a plurality of frame echocardiogram images, the any one frame of echocardiogram image is input into a semi-supervised depth segmentation model to calculate a depth value for each pixel point by using the semi-supervised depth segmentation model, and a relative depth estimation value corresponding to each pixel point is obtained.

[0072] According to an embodiment of the present invention, according to a second predetermined rule, relative depth estimation values corresponding to each pixel point located in the left ventricular region are selected from a plurality of relative depth estimation values.

[0073] According to an embodiment of the present invention, the second predetermined rule indicates that when the relative depth estimation value is 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in a region outside the left ventricle, and when the relative depth estimation value is not 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in a region inside the left ventricle.

[0074] For example, when the relative depth estimation values of pixel points in the regions of [20:20], [20:40], [40:20], and [40:40] in the echocardiogram image are all greater than 0, that is, when the relative depth estimation value is not 0, it is confirmed that all pixel points in this region are located in the left ventricular region, and relative depth estimation values corresponding to each of all pixel points located in the left ventricular region are obtained.

[0075] According to an embodiment of the present invention, the above operations are performed on each of a plurality of frame echocardiogram images to obtain relative depth estimation values corresponding to each pixel point located in the left ventricular region in each frame of echocardiogram image.

[0076] According to an embodiment of the present invention, for each frame of echocardiogram image in the echocardiogram video input into the semi-supervised depth segmentation model, a relative depth estimation value of each pixel point is first obtained, and then according to whether the relative depth estimation value is greater than 0, it is determined whether the pixel point is located in the left ventricular region, so as to obtain the depth value, that is, the relative depth estimation value, of each pixel point located in the left ventricular region in each frame of echocardiogram image.

[0077] According to an embodiment of the present invention, by inputting an echocardiogram video including multiple frames of echocardiogram images into a semi-supervised depth segmentation model, the semi-supervised depth segmentation model calculates the depth value for each pixel point in each frame of the echocardiogram image to obtain a relative depth estimation value corresponding to each pixel point. Based on a second predetermined rule, according to the relative depth estimation value, it is determined whether the relative depth estimation value is equal to 0. In the case where the relative depth estimation value is not equal to 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in the left ventricular region, so as to obtain the relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of the echocardiogram image. It realizes the division of regions for each frame of echocardiogram image in the echocardiogram video by using a semi-supervised depth segmentation model with high robustness, clearly and accurately obtains the region of the left ventricle in each frame of the echocardiogram image, and obtains the relative depth estimation value corresponding to each pixel point located in the left ventricular region, so as to calculate the volume of the left ventricle according to the relative depth estimation values of the pixel points in the left ventricular region and improve the estimation efficiency.

[0078] According to an embodiment of the present invention, the target electromagnetic echo signal includes electromagnetic echo signals corresponding to each spatial point in a three-dimensional space.

[0079] According to an embodiment of the present invention, the method for calculating the candidate volume of the left ventricle in each frame of the echocardiogram image based on multiple relative depth estimation values in each frame of the echocardiogram image includes the following operations.

[0080] According to an embodiment of the present invention, the relative depth estimation values of each pixel point in each frame of the echocardiogram image are summed to calculate the candidate volume of the left ventricle in each frame of the echocardiogram image.

[0081] For example, there are 6 pixel points in the left ventricular region of the echocardiogram image. The relative depth estimation value corresponding to the first pixel point is 2, the relative depth estimation value corresponding to the second pixel point is 13, the relative depth estimation value corresponding to the third pixel point is 9, the relative depth estimation value corresponding to the fourth pixel point is 4, the relative depth estimation value corresponding to the fifth pixel point is 0.71, and the relative depth estimation value corresponding to the sixth pixel point is 0.8. Then, by summing the above 6 relative depth estimation values, the candidate volume of the left ventricle in this frame of the echocardiogram image can be 29.51.

[0082] According to an embodiment of the present invention, by summing the relative depth estimation values of each pixel point in each frame of the echocardiogram image, the candidate volume of the left ventricle in each frame of the echocardiogram image can be obtained, realizing the accurate calculation of the volume of the left ventricle in each frame of the echocardiogram image in the video, so as to obtain the accurate ejection fraction of the left ventricle according to the candidate volume of the left ventricle.

[0083] According to an embodiment of the present invention, a method for determining a plurality of target systolic volumes and a plurality of target diastolic volumes from a plurality of candidate volumes based on a first predetermined rule and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes includes the following operations.

[0084] According to an embodiment of the present invention, an initial volume curve graph of the left ventricle is generated based on a plurality of candidate volumes.

[0085] According to an embodiment of the present invention, each frame of echocardiogram image corresponds to a candidate volume of the left ventricle, so that the echocardiogram video corresponds to a plurality of candidate volumes. An initial volume curve graph corresponding to the left ventricle is generated based on the plurality of candidate volumes. The initial volume curve graph includes a plurality of minimum points, a plurality of maximum points, and other numerical points. Among them, the minimum points and the maximum points can respectively correspond to the states of target systole and target diastole. For example, the motion states of cardiac systole and cardiac diastole.

[0086] According to an embodiment of the present invention, a peak seeking function is used to perform peak seeking processing on the initial volume curve graph to determine a plurality of target systolic volumes and a plurality of target diastolic volumes.

[0087] According to an embodiment of the present invention, a plurality of minimum points and a plurality of maximum points are captured from the initial volume curve graph by using a peak seeking function. The candidate volume corresponding to the minimum point is confirmed as the target systolic volume, the echocardiogram image corresponding to the candidate volume of the minimum point is the end-systolic frame, the candidate volume corresponding to the maximum point is the target diastolic volume, and the echocardiogram image corresponding to the candidate volume of the maximum point is the end-diastolic frame, so as to obtain a plurality of target systolic volumes and a plurality of target diastolic volumes.

[0088] According to an embodiment of the present invention, a target ejection function evaluation value is calculated based on the plurality of target systolic volumes and the plurality of target diastolic volumes.

[0089] According to an embodiment of the present invention, the beats of the heartbeat, that is, the cardiac cycle, can be divided according to the plurality of target systolic volumes and the plurality of target diastolic volumes. According to the target systolic volume and the target diastolic volume within each cardiac cycle, a target ejection function evaluation value of the target left ventricle can be obtained, so as to assist relevant professionals in targeted diagnosis as an intermediate result.

[0090] According to an embodiment of the present invention, by constructing an initial volume curve graph of the left ventricle based on the obtained multiple candidate volumes of the left ventricle, using a peak seeking algorithm to determine multiple minimum points from the initial volume curve graph as the target systolic volume and the target diastolic volume, and at the same time, multiple systolic end frames corresponding to the target systolic volume and diastolic end frames corresponding to the target diastolic volume can be determined. According to the obtained multiple target systolic volumes and multiple target diastolic volumes, a target ejection function evaluation value of the target left ventricle can be calculated. It realizes the use of the candidate volume of the left ventricle obtained based on the semi-supervised depth segmentation model, so as to obtain an accurate target ejection function evaluation value of the left ventricle. Moreover, while obtaining the target ejection function evaluation value of the left ventricle, parameters such as the initial volume curve graph of the left ventricle, diastolic end frame, systolic end frame, target systolic volume, and target diastolic volume can be separately extracted for reference, so that relevant personnel can verify any value in the intermediate process and check and adjust it in a timely manner.

[0091] According to an embodiment of the present invention, the method for calculating the target ejection function evaluation value based on multiple target systolic volumes and multiple target diastolic volumes includes the following operations.

[0092] According to an embodiment of the present invention, based on the cardiac cycle division rule, the initial volume curve graph is processed for cardiac cycle division to obtain a target volume curve graph.

[0093] According to an embodiment of the present invention, the target volume curve graph represents a curve graph divided into multiple cardiac cycles, and the cardiac cycle division rule represents dividing with an adjacent target systolic volume and a target diastolic volume as a cardiac cycle.

[0094] According to an embodiment of the present invention, the cardiac cycle division rule represents dividing with an adjacent target systolic volume and a target diastolic volume as a cardiac cycle, that is, each cardiac cycle includes a diastolic end frame and a systolic end frame.

[0095] According to an embodiment of the present invention, ejection calculation is performed on the target systolic volume and target diastolic volume in each cardiac cycle to obtain an ejection fraction corresponding to each cardiac cycle.

[0096] According to an embodiment of the present invention, the ejection fraction can be calculated according to formula (1).

[0097] (1)

[0098] Wherein, EF can represent the ejection fraction corresponding to a cardiac cycle, EDV can represent the target diastolic volume within a cardiac cycle, and ESV can represent the target systolic volume within a cardiac cycle.

[0099] For example, according to the cardiac cycle division rule, the initial volume curve graph is divided into 6 cardiac beats, that is, 6 cardiac cycles. The target systolic volume in the first cardiac cycle is 1 L, and the target diastolic volume is 3 L. The target systolic volume in the second cardiac cycle is 0.5 L, and the target diastolic volume is 3.5 L. The target systolic volume in the third cardiac cycle is 1 L, and the target diastolic volume is 4 L. The target systolic volume in the fourth cardiac cycle is 2 L, and the target diastolic volume is 3 L. The target systolic volume in the fifth cardiac cycle is 2 L, and the target diastolic volume is 4 L. The target systolic volume in the sixth cardiac cycle is 2.5 L, and the target diastolic volume is 3.5 L. Then the ejection fraction corresponding to the first cardiac cycle is 66.7%, the ejection fraction corresponding to the second cardiac cycle is 85.7%, the ejection fraction corresponding to the third cardiac cycle is 75%, the ejection fraction corresponding to the fourth cardiac cycle is 33.3%, the ejection fraction corresponding to the fifth cardiac cycle is 50%, and the ejection fraction corresponding to the sixth cardiac cycle is 28.6%.

[0100] According to an embodiment of the present invention, multiple ejection fractions are averaged to obtain a target ejection function evaluation value.

[0101] For example, the ejection fraction corresponding to the first cardiac cycle is 66.7%, the ejection fraction corresponding to the second cardiac cycle is 85.7%, the ejection fraction corresponding to the third cardiac cycle is 75%, the ejection fraction corresponding to the fourth cardiac cycle is 33.3%, the ejection fraction corresponding to the fifth cardiac cycle is 50%, and the ejection fraction corresponding to the sixth cardiac cycle is 28.6%. Then the target ejection function evaluation value of the left ventricle of the target object is 56.55%.

[0102] According to an embodiment of the present invention, it is also possible to randomly select the ejection fractions corresponding to any number of cardiac cycles from multiple cardiac cycles for averaging calculation to obtain a target ejection function evaluation value. For example, from 6 cardiac cycles, the ejection fraction corresponding to the first cardiac cycle, the ejection fraction corresponding to the fourth cardiac cycle, and the ejection fraction corresponding to the sixth cardiac cycle are selected for averaging to obtain a target ejection function evaluation value.

[0103] According to an embodiment of the present invention, by performing cardiac cycle division processing on the initial volume curve graph based on a cardiac cycle division rule, a target volume curve graph including multiple cardiac cycles is obtained. Each cardiac cycle only includes one target diastolic volume and one target systolic volume. The ejection fraction is calculated based on one target diastolic volume and one target systolic volume in each cardiac cycle to obtain an ejection fraction corresponding to each cardiac cycle. Then, the average value of multiple ejection fractions is calculated to obtain a target ejection function evaluation value of the left ventricle of the target object, realizing obtaining an average ejection fraction value based on multiple cardiac cycles to improve the accuracy of the target ejection function evaluation value and facilitating subsequent operations based on the target ejection function evaluation value.

[0104] According to an embodiment of the present invention, the semi-supervised deep segmentation model is trained by the following method, and the training method includes the following operations.

[0105] According to an embodiment of the present invention, a training data set is obtained. The training data set includes an echocardiogram sample video, which is composed of multiple frames of echocardiogram sample images. Among the multiple frames of echocardiogram sample images, there are diastolic sample images with segmentation labels, systolic sample images with segmentation labels, and echocardiogram sample images without segmentation labels.

[0106] According to an embodiment of the present invention, the echocardiogram sample video in the training data set can be saved in three channels of red, green, and blue (RGB). The echocardiogram sample video in the training data set can be preprocessed to crop the echocardiogram sample video into a sample video of 112×112 pixels, so as to facilitate the semi-supervised deep segmentation model to be trained to process and train the sample video.

[0107] According to an embodiment of the present invention, pre-segmentation processing is performed on the echocardiogram sample video to obtain a pre-depth segmentation image corresponding to each frame of echocardiogram sample image in the echocardiogram sample video.

[0108] According to an embodiment of the present invention, the pre-depth segmentation image includes a pre-depth segmentation label corresponding to each pixel point, and the pre-depth segmentation label represents the depth of each pixel point in the left ventricle region.

[0109] According to an embodiment of the present invention, pre-segmentation processing can be performed on the diastolic sample images and systolic sample images with segmentation labels in the echocardiogram sample video to obtain a pre-depth segmentation image corresponding to the diastolic sample images and a pre-depth segmentation image corresponding to the systolic sample images.

[0110] According to an embodiment of the present invention, the long axis of the left ventricle in the diastolic sample image and the systolic sample image can be determined first. According to the long axis of the left ventricle, a plurality of cross-sections perpendicular to the long axis of the left ventricle can be selected. The width of each cross-section is equivalent to the width of the left ventricle. Since the cross-sections in the left ventricle of the diastolic sample image and the systolic sample image are not perpendicular to the coordinate system, the diastolic sample image and the systolic sample image can be rotated to facilitate the calculation of the depth value corresponding to each pixel point. The rotation angle is shown in formula (2). Among them, the relationship between the depth value corresponding to the pixel point and the coordinates of the pixel point in the unrotated coordinate system can be shown in formula (3).

[0111] (2)

[0112] Among them, can be characterized as the coordinate system perpendicular to the cross-section after rotation, can be characterized as the rotation angle, can be characterized as the coordinate system before rotation. u can be characterized as the horizontal axis in the coordinate system perpendicular to the cross-section after rotation, v can be characterized as the vertical axis in the coordinate system perpendicular to the cross-section after rotation, x can be characterized as the horizontal axis in the coordinate system before rotation, and y can be characterized as the vertical axis in the coordinate system before rotation.

[0113] (3)

[0114] Among them, z i can be characterized as the depth value corresponding to the i-th pixel point, x i can be characterized as the abscissa of the i-th pixel point, y i can be characterized as the ordinate of the i-th pixel point.

[0115] According to an embodiment of the present invention, the left ventricle can be regarded as an ellipsoid, the length R of the cross-section in the left ventricle is regarded as the diameter of the disk of the left ventricle, and the center of each cross-section is the center of the disk. After rotating the diastolic sample image and the systolic sample image, the radius r from the pixel point to the center of the cross-section can be determined according to the coordinates of the pixel point.

[0116] According to an embodiment of the present invention, when the radius r from the pixel point to the center of the cross-section is greater than the length R of the cross-section, the pixel point is located in the area outside the left ventricle, and the depth segmentation label of the pixel point located in the area outside the left ventricle is preset to 0. When the radius r from the pixel point to the center of the cross-section is less than or equal to the length R of the cross-section, the pixel point is located in the area inside the left ventricle. Using the Pythagorean theorem, according to the radius r from the pixel point to the center of the cross-section and the length R of the cross-section, the depth value corresponding to each pixel point, that is, the depth segmentation label corresponding to each pixel point, is calculated.

[0117] According to an embodiment of the present invention, through the above-mentioned preprocessing, pre-depth segmentation labels corresponding to each pixel point in the diastolic sample image and the systolic sample image can be obtained.

[0118] According to an embodiment of the present invention, the echocardiogram sample video is input into a semi-supervised depth segmentation model to be trained for segmentation processing, and a segmentation training image corresponding to each frame of echocardiogram sample image in the echocardiogram sample video is obtained.

[0119] According to an embodiment of the present invention, the semi-supervised depth segmentation model to be trained is composed of a first image segmentation network and a second image segmentation network with the same initial weights, and the segmentation training images include diastolic training images, systolic training images, and multiple frames of cardiac motion training images.

[0120] According to an embodiment of the present invention, the first image segmentation network and the second image segmentation network with the same initial weights can be represented according to formulas (4) and (5).

[0121] (4)

[0122] Wherein, P 1 can be characterized as the first image segmentation network, X can be characterized as the echocardiogram sample image input into the first image segmentation network, can be characterized as the weight of the first image segmentation network.

[0123] (5)

[0124] Wherein, P 2 can be characterized as the second image segmentation network, X can be characterized as the echocardiogram sample image input into the first image segmentation network, can be characterized as the weight of the second image segmentation network.

[0125] According to an embodiment of the present invention, before training the semi-supervised depth segmentation model, the initial weights of the two first image segmentation networks and the second image segmentation network can be the same, that is equal to .

[0126] According to an embodiment of the present invention, based on the loss function, a training loss value is obtained according to multiple frames of pre-depth segmentation images, diastolic training images, systolic training images, and multiple frames of cardiac motion training images.

[0127] According to an embodiment of the present invention, according to the training loss value, by adjusting the initial weights of the first image segmentation network and the second image segmentation network, a trained semi-supervised depth segmentation model is obtained.

[0128] According to an embodiment of the present invention, by obtaining a training data set, pre-segmenting an echocardiogram sample video in the training data set, pre-depth segmentation images corresponding to each frame of echocardiogram sample images in the echocardiogram sample video are obtained. At the same time, the echocardiogram sample video is input into a semi-supervised depth segmentation model to be trained for segmentation processing, and segmentation training images corresponding to each frame of echocardiogram sample images in the echocardiogram sample video are obtained. Based on a loss function, according to multiple frames of pre-depth segmentation images, diastolic training images, systolic training images, and multiple frames of cardiac training images, a training loss value is obtained. According to the training loss value, by adjusting the initial weights of the first image segmentation network and the initial weights of the second image segmentation network, a trained semi-supervised depth segmentation model is obtained. The training of the semi-supervised depth segmentation model is realized, so as to obtain a semi-supervised depth segmentation model with high robustness. Using the trained semi-supervised depth segmentation model to process an echocardiogram video, an evaluation value of the target ejection function of the left ventricle with high accuracy can be directly output, improving the processing efficiency.

[0129] According to an embodiment of the present invention, the diastolic training images include a first diastolic training image processed by the first image segmentation network and a second diastolic training image processed by the second image segmentation network. The systolic training images include a first systolic training image processed by the first image segmentation network and a second systolic training image processed by the second image segmentation network. The first diastolic training image and the second diastolic training image include diastolic pixel points, and the first systolic training image and the second systolic training image include systolic pixel points.

[0130] According to an embodiment of the present invention, the method for obtaining a training loss value according to multiple frames of pre-depth segmentation images, diastolic training images, systolic training images, and multiple frames of cardiac training images based on a loss function includes the following operations.

[0131] According to an embodiment of the present invention, for each diastolic pixel point in the first diastolic training image, according to the training depth of the diastolic pixel point and the pre-depth segmentation label of the first target pixel point, a diastolic loss function for each of the multiple diastolic pixel points in the first diastolic training image is obtained.

[0132] According to an embodiment of the present invention, the first target pixel point is determined from a first pre-depth segmentation image corresponding to the diastolic training image according to the coordinate information of the diastolic pixel point.

[0133] According to an embodiment of the present invention, two frames of echocardiogram sample images with segmentation labels are included in the multiple frames of echocardiogram sample images, that is, a diastolic sample image with a segmentation label and a systolic sample image with a segmentation label.

[0134] According to an embodiment of the present invention, a diastolic sample image containing segmentation labels can be input into a first image segmentation network to obtain a first diastolic training image. A first pre-depth segmentation image corresponding to the diastolic training image is determined from multiple pre-depth segmentation images. Then, the coordinate information of diastolic pixel points is obtained from the first diastolic training image. According to the coordinate information of the diastolic pixel points, first target pixel points are determined from the first pre-depth segmentation image. Finally, based on the training depth of the diastolic pixel points and the pre-depth segmentation labels of the first target pixel points, mean square error calculation is performed to obtain the diastolic loss function of the diastolic pixel points. The above operations are performed on each diastolic pixel point in the first diastolic training image to obtain the diastolic loss functions of multiple diastolic pixel points respectively.

[0135] According to an embodiment of the present invention, for each systolic pixel point in the second systolic training image, based on the training depth of the systolic pixel point and the pre-depth segmentation labels of the second target pixel points, the systolic loss functions of multiple systolic pixel points in the second systolic training image are obtained respectively.

[0136] According to an embodiment of the present invention, the second target pixel points are determined from a second pre-depth segmentation image corresponding to the systolic training image according to the coordinate information of the systolic pixel points.

[0137] According to an embodiment of the present invention, a systolic sample image containing segmentation labels can be input into a second image segmentation network to obtain a second systolic training image. A second pre-depth segmentation image corresponding to the systolic training image is determined from multiple pre-depth segmentation images. Then, the coordinate information of systolic pixel points is obtained from the second systolic training image. According to the coordinate information of the systolic pixel points, second target pixel points are determined from the second pre-depth segmentation image. Finally, based on the training depth of the systolic pixel points and the pre-depth segmentation labels of the second target pixel points, mean square error calculation is performed to obtain the systolic loss function of the systolic pixel points. The above operations are performed on each systolic pixel point in the second systolic training image to obtain the systolic loss functions of multiple systolic pixel points respectively.

[0138] According to an embodiment of the present invention, based on the supervision loss value determined based on multiple diastolic loss functions, multiple systolic loss functions, the size information of the diastolic training image, and the size information of the systolic training image, and the target cross pseudo-supervision loss value determined based on the size information of multiple frame cardiac training images, the diastolic training image, the systolic training image, and multiple frame pre-depth segmentation images, the training loss value is determined.

[0139] According to an embodiment of the present invention, since the diastolic training image corresponds to the diastolic sample image, the systolic training image corresponds to the systolic sample image, and both the diastolic sample image and the systolic sample image are obtained from an echocardiogram sample video, the size information of the diastolic training image and the size information of the systolic training image can be the same.

[0140] According to an embodiment of the present invention, the supervised loss value can be calculated according to formula (6).

[0141] (6)

[0142] Wherein, can be characterized as the supervised loss value, can be characterized as multi-frame ultrasound sample images containing segmentation labels, that is, diastolic sample images containing segmentation labels and systolic sample images containing segmentation labels. X can be characterized as the Xth ultrasound sample image containing segmentation labels, that is, a diastolic sample image containing segmentation labels or a systolic sample image containing segmentation labels. W can be characterized as the length information of the ultrasound sample image, H can be characterized as the width information of the ultrasound sample image, and i can be characterized as the ith pixel point, that is, the ith diastolic pixel point, p li can be characterized as the training depth of the ith diastolic pixel point in the first diastolic training image, p si can be characterized as the training depth of the ith systolic pixel point in the second systolic training image, y * li can be characterized as the pre-depth segmentation label of the first target pixel point, y * si can be characterized as the pre-depth segmentation label of the second target pixel point, can be characterized as the diastolic loss function of the ith diastolic pixel point in the first diastolic training image, can be characterized as the systolic loss function of the ith systolic pixel point in the second systolic training image.

[0143] According to an embodiment of the present invention, the training loss value can be calculated according to formula (7).

[0144] (7)

[0145] Wherein, can be characterized as the training loss value, can be characterized as the supervised loss value, can be characterized as the target cross pseudo-supervised loss value, can be characterized as the balance weight.

[0146] According to an embodiment of the present invention, for each diastolic pixel point in the first diastolic training image, based on the training depth of the diastolic pixel point and the pre-depth segmentation label of the first target pixel point, a mean square error calculation is performed to obtain the respective diastolic loss functions of multiple diastolic pixel points in the first diastolic training image. For each systolic pixel point in the second systolic training image, based on the training depth of the systolic pixel point and the pre-depth segmentation label of the second target pixel point, a mean square error calculation is performed to obtain the respective systolic loss functions of multiple systolic pixel points in the second systolic training image. According to the supervision loss value determined based on multiple diastolic loss functions, multiple systolic loss functions, the size information of the diastolic training image, and the size information of the systolic training image, and then according to the supervision loss value and the target cross pseudo-supervision loss value, the training loss value is determined, thereby realizing the training of the semi-supervised depth segmentation model to be trained. The loss value is calculated using the training result output by the semi-supervised depth segmentation model and the result in the pre-depth segmented image, so as to improve the accuracy of the depth segmentation label output of the first image segmentation network and the first image segmentation network for pixel points, and improve the robustness of the semi-supervised depth segmentation model, so as to directly apply the trained model to the process of detecting the object to be measured and improve the detection efficiency.

[0147] According to an embodiment of the present invention, the cardiac training image includes a first network training image processed by a first image segmentation network and a second network training image processed by a second image segmentation network. The first network training image includes first cardiac pixel points, and the second network training image includes second cardiac pixel points.

[0148] According to an embodiment of the present invention, the target cross pseudo-supervision loss value is determined by the following method, which specifically includes the following operations.

[0149] According to an embodiment of the present invention, based on the training depth of each first cardiac pixel point and the training depth of the third target pixel point, the respective first mean square error functions of multiple first cardiac pixel points in multiple first network training images are determined.

[0150] According to an embodiment of the present invention, the third target pixel point is determined from the second network training image according to the coordinate information of the first cardiac pixel point.

[0151] According to an embodiment of the present invention, an echocardiogram sample image without segmentation labels can be input into a first image segmentation network and a second image segmentation network simultaneously to obtain a first network training image and a second network training image respectively. Then, the coordinate information of the first cardiac pixels can be obtained from the first network training image. According to the coordinate information of the first cardiac pixels, the third target pixels can be determined from the second network training image. Finally, according to the training depth of the first cardiac pixels and the training depth of the third target pixels, mean square error calculation is performed to obtain the first mean square error function of the first cardiac pixels. The above operations are performed on each first cardiac pixel in the first network training image to obtain the first mean square error functions of multiple first cardiac pixels respectively.

[0152] According to an embodiment of the present invention, according to the training depth of each second cardiac pixel and the training depth of the fourth target pixels, the second mean square error functions of multiple second cardiac pixels in multiple second network training images are determined respectively.

[0153] According to an embodiment of the present invention, the fourth target pixels are determined from the first network training image according to the coordinate information of the second cardiac pixels.

[0154] According to an embodiment of the present invention, an echocardiogram sample image without segmentation labels can be input into a first image segmentation network and a second image segmentation network simultaneously to obtain a first network training image and a second network training image respectively. Then, the coordinate information of the second cardiac pixels can be obtained from the second network training image. According to the coordinate information of the second cardiac pixels, the fourth target pixels can be determined from the first network training image. Finally, according to the training depth of the second cardiac pixels and the training depth of the fourth target pixels, mean square error calculation is performed to obtain the second mean square error function of the second cardiac pixels. The above operations are performed on each second cardiac pixel in the second network training image to obtain the second mean square error functions of multiple second cardiac pixels respectively.

[0155] According to an embodiment of the present invention, each frame of the echocardiogram sample image without segmentation labels can be processed as described above to obtain multiple first mean square error functions and multiple second mean square error functions corresponding to each frame of the echocardiogram sample image.

[0156] According to an embodiment of the present invention, both the diastolic training image and the systolic training image are processed to obtain the third mean square error functions of multiple diastolic pixels in the first diastolic training image, the fourth mean square error functions of multiple diastolic pixels in the second diastolic training image, the fifth mean square error functions of multiple systolic pixels in the first systolic training image, and the sixth mean square error functions of multiple systolic pixels in the second systolic training image.

[0157] According to an embodiment of the present invention, the diastolic training image can be first input into the first image segmentation network and the second image segmentation network simultaneously to obtain the first diastolic training image and the second diastolic training image respectively. Then, the coordinate information of the diastolic pixel points can be obtained from the first diastolic training image. According to the coordinate information of the diastolic pixel points, the fifth target pixel points can be determined from the second diastolic training image. According to the training depth of the diastolic pixel points in the first diastolic training image and the training depth of the fifth target pixel points, mean square error calculation is performed to obtain the third mean square error function of the diastolic pixel points in the first diastolic training image. The above operations are performed on each diastolic pixel point in the first diastolic training image to obtain the third mean square error function of each of the multiple diastolic pixel points. At the same time, the coordinate information of the diastolic pixel points is obtained from the second diastolic training image. According to the coordinate information of the diastolic pixel points, the sixth target pixel points are determined from the first diastolic training image. According to the training depth of the diastolic pixel points in the second diastolic training image and the training depth of the sixth target pixel points, mean square error calculation is performed to obtain the fourth mean square error function of the diastolic pixel points in the second diastolic training image. The above operations are performed on each diastolic pixel point in the second diastolic training image to obtain the fourth mean square error function of each of the multiple diastolic pixel points.

[0158] According to an embodiment of the present invention, the systolic training image can be first input into the first image segmentation network and the second image segmentation network simultaneously to obtain the first systolic training image and the second systolic training image respectively. Then, the coordinate information of the systolic pixel points can be obtained from the first systolic training image. According to the coordinate information of the systolic pixel points, the seventh target pixel points can be determined from the second systolic training image. According to the training depth of the systolic pixel points in the first systolic training image and the training depth of the seventh target pixel points, mean square error calculation is performed to obtain the fifth mean square error function of the systolic pixel points in the first systolic training image. The above operations are performed on each systolic pixel point in the first systolic training image to obtain the fifth mean square error function of each of the multiple systolic pixel points. At the same time, the coordinate information of the systolic pixel points is obtained from the second systolic training image. According to the coordinate information of the systolic pixel points, the eighth target pixel points are determined from the first systolic training image. According to the training depth of the systolic pixel points in the second systolic training image and the training depth of the eighth target pixel points, mean square error calculation is performed to obtain the sixth mean square error function of the systolic pixel points in the second systolic training image. The above operations are performed on each systolic pixel point in the second systolic training image to obtain the sixth mean square error function of each of the multiple systolic pixel points.

[0159] According to an embodiment of the present invention, based on multiple first mean square error functions, multiple second mean square error functions, and the size information of multiple frames of cardiac motion training images, the first cross-pseudo-supervision loss value is determined.

[0160] According to an embodiment of the present invention, since the echocardiogram sample image, diastolic sample image, and systolic sample image without segmentation labels are all obtained from the echocardiogram sample video, the size information of the echocardiogram sample image without segmentation labels can be the same as that of the diastolic training image, systolic training image, and cardiac training image, and the size information of each frame of the multi-frame ultrasound sample images can be the same.

[0161] According to an embodiment of the present invention, the first cross pseudo-supervision loss value can be calculated according to formula (8).

[0162] (8)

[0163] Wherein, can be characterized as the first cross pseudo-supervision loss value, can be characterized as multi-frame echocardiogram sample images without segmentation labels, u can be characterized as the u-th multi-frame echocardiogram sample image without segmentation labels, W can be characterized as the length information of the ultrasound sample image, H can be characterized as the width information of the ultrasound sample image, i can be characterized as the i-th pixel point, that is, the i-th cardiac pixel point, can be characterized as the training depth of the i-th cardiac pixel point in the first network training image, can be characterized as the training depth of the i-th cardiac pixel point in the second network training image, can be characterized as the training depth of the third target pixel point corresponding to the i-th cardiac pixel point in the second network training image, can be characterized as the training depth of the fourth target pixel point corresponding to the i-th cardiac pixel point in the first network training image, can be characterized as the first mean square error function of the i-th cardiac pixel point, can be characterized as the second mean square error function of the i-th cardiac pixel point.

[0164] According to an embodiment of the present invention, based on multiple third mean square error functions, multiple fourth mean square error functions, multiple fifth mean square error functions, multiple sixth mean square error functions, the size information of the diastolic training image, and the size information of the systolic training image, the second cross pseudo-supervision loss value is determined.

[0165] According to an embodiment of the present invention, based on the first cross pseudo-supervision loss value and the second cross pseudo-supervision loss value, the target cross pseudo-supervision loss value is determined.

[0166] According to an embodiment of the present invention, the target cross pseudo-supervision loss value can be calculated according to formula (9).

[0167] (9)

[0168] Among them, it can be characterized as a target cross pseudo - supervision loss value, it can be characterized as a second cross pseudo - supervision loss value, it can be characterized as a first cross pseudo - supervision loss value.

[0169] According to an embodiment of the present invention, the second cross pseudo - supervision loss value is first calculated according to a plurality of third mean square error functions corresponding to diastolic training images, a plurality of fourth mean square error functions, and the size information of the diastolic training images according to the above - mentioned first cross pseudo - supervision loss value to obtain a diastolic cross pseudo - supervision loss value. Then, according to a plurality of fifth mean square error functions corresponding to systolic training images, a plurality of sixth mean square error functions, and the size information of the systolic training images according to the above - mentioned first cross pseudo - supervision loss value, a systolic cross pseudo - supervision loss value is obtained. The diastolic cross pseudo - supervision loss value and the systolic cross pseudo - supervision loss value are added together to obtain the second cross pseudo - supervision loss value.

[0170] According to an embodiment of the present invention, by inputting cardiac motion training images into a first image segmentation network and a second image segmentation network, a first network training image and a second network training image are respectively obtained. For the first network training image, using the training depth obtained by training with the second network training image as a reference, a first mean square error function is constructed with the training depth in the first network training image. For the second network training image, using the training depth obtained by training with the first network training image as a reference, a second mean square error function is constructed with the training depth in the second network training image. Thus, a first cross pseudo - supervision loss value is obtained according to the first mean square error function and the second mean square error function. The above operations are also performed on the diastolic training images and the systolic training images to obtain a second cross pseudo - supervision loss value. The first cross pseudo - supervision loss value and the second cross pseudo - supervision loss value are added together to obtain a target cross pseudo - supervision loss value, which realizes cross - training of the semi - supervised depth segmentation model to be trained using images without segmentation labels, so as to improve the accuracy of the depth segmentation labels output by the semi - supervised depth segmentation model. And, while performing cross - training using sample images without segmentation labels, randomly grabbing sample images with segmentation labels is also processed as above to increase the randomness of the input data, further improving the accuracy of the depth segmentation labels output by the semi - supervised depth segmentation model and the high robustness of the semi - supervised depth segmentation model, so as to directly apply the trained model to the process of detecting the object to be measured and improve the detection efficiency.

[0171] Figure 3 It shows a schematic diagram of the left ventricle segmented in an echocardiogram video according to an embodiment of the present invention.

[0172] As Figure 3 shown, Figure 3The schematic diagram after the left ventricle is segmented is shown. The left image is an ultrasound image. After magnifying the left ventricle, it can be seen that the left ventricle can be approximated as a cylinder and is segmented into multiple cross-sections. The multiple cross-sections are parallel to each other, and the axis perpendicular to the multiple cross-sections can be the long axis of the left ventricle.

[0173] Figure 4a The schematic diagram of the regional heat of the left ventricle segmented by using the semi-supervised depth segmentation model according to an embodiment of the present invention is shown. Figure 4b The schematic diagram of the regional heat curve of the left ventricle segmented by using the semi-supervised depth segmentation model according to an embodiment of the present invention is shown.

[0174] As Figures 4a to 4b shown, Figures 4a to 4b The schematic diagram of the regional heat and the regional heat curve graph of the left ventricle segmented by using the semi-supervised depth segmentation model of the present invention is shown. It can be seen from the figure that through the method of the present invention, the left ventricle can be clearly located, so as to calculate the volume within the left ventricle, and thus obtain the target ejection function evaluation value of the left ventricle.

[0175] Figure 5 The schematic diagram of the process for obtaining the supervised loss value according to an embodiment of the present invention is shown.

[0176] As Figure 5 shown, Figure 5 The process for obtaining the supervised loss value of the present invention is shown. X l can be characterized as a diastolic training image, X s can be characterized as a systolic training image, can be characterized as the first image segmentation network, can be characterized as the second image segmentation network, P l can be characterized as the first diastolic training image, P s can be characterized as the second systolic training image, P l Output can be characterized as a diastolic loss function, P s Output can be characterized as a systolic loss function, Y 1 * can be characterized as the first pre-depth segmentation image, Y 2 * can be characterized as the second pre-depth segmentation image, It can be characterized as a supervised loss value. Input the diastolic training image into the first image segmentation network to obtain the first diastolic training image, input the systolic training image into the second image segmentation network to obtain the second systolic training image, select the first pre-depth segmentation image corresponding to the diastolic training image and the second pre-depth segmentation image corresponding to the systolic training image from multiple frames of pre-depth segmentation images, construct a diastolic loss function according to the pre-depth segmentation label in the first pre-depth segmentation image and the training depth in the first diastolic training image, construct a systolic loss function according to the pre-depth segmentation label in the second pre-depth segmentation image and the training depth in the second systolic training image, and obtain the supervised loss value according to the systolic loss function and the diastolic loss function.

[0177] Figure 6 FIG. shows a schematic diagram of a process for obtaining a first cross pseudo-supervised loss value according to an embodiment of the present invention.

[0178] As Figure 6 shown, Figure 6 FIG. shows the process of the present invention for obtaining the first cross pseudo-supervised loss value. It can be characterized as an echocardiogram sample image without a segmentation label. It can be characterized as the first image segmentation network. It can be characterized as the second image segmentation network, P 1 It can be characterized as the first network training image, P 2 It can be characterized as the second network training image, Y 1 It can be characterized as the second network training image used for mean square error function calculation with the first network training image, Y 2 It can be characterized as the first network training image used for mean square error function calculation with the second network training image. Input the echocardiogram sample image without a segmentation label into the first image segmentation network and the second image segmentation network to obtain the first network training image and the second network training image respectively. Take the second network training image as the reference image for the first mean square error function calculation with the first network training image, take the first network training image as the reference image for the second mean square error function calculation with the second network training image, construct the first mean square error function according to the training depth in the first network training image and the training depth in the second network training image, construct the second mean square error function according to the training depth in the second network training image and the training depth in the first network training image, and obtain the first cross pseudo-supervised loss value according to the first mean square error function and the second mean square error function.

[0179] Figure 7 FIG. shows a schematic diagram of the comparison between the ejection fraction obtained by using the model of the present invention and the true value according to an embodiment of the present invention.

[0180] AsFigure 7 As shown Figure 7 Figure 7 shows the comparison between the ejection fraction obtained by the method of the present invention and the true value. The abscissa can be characterized as the true value of the ejection fraction, and the ordinate can be characterized as the measured ejection fraction obtained by the method of the present invention. It can be seen from the figure that there is a high consistency between the measured ejection fraction obtained by the method of the present invention and the true value.

[0181] Figure 8 Figure 8 shows a structural block diagram of a device for evaluating the ejection function value of a semi-supervised depth segmentation model according to an embodiment of the present invention.

[0182] As Figure 8 Figure 8 shows, the device for evaluating the ejection function value based on the semi-supervised depth segmentation model of this embodiment includes: an acquisition module 810, a segmentation module 820, a calculation module 830, and a obtaining module 840.

[0183] The acquisition module 810 is configured to acquire an echocardiogram video, where the echocardiogram video includes multiple frames of echocardiogram images. The acquisition module 810 can be used to perform the operation S210 described above, which will not be elaborated here.

[0184] The segmentation module 820 is configured to perform segmentation processing on the echocardiogram video by using a semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricular region in each frame of echocardiogram image, where the semi-supervised depth segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value characterizes the depth of each pixel point in the left ventricular region. The segmentation module 820 can be used to perform the operation S220 described above, which will not be elaborated here.

[0185] The calculation module 830 is configured to calculate, according to multiple relative depth estimation values in each frame of echocardiogram image, a candidate volume of the left ventricle in each frame of echocardiogram image. The calculation module 830 can be used to perform the operation S230 described above, which will not be elaborated here.

[0186] The obtaining module 840 is configured to determine multiple target systolic volumes and multiple target diastolic volumes from multiple candidate volumes based on a first predetermined rule, and obtain a target ejection function evaluation value of the left ventricle according to the multiple target systolic volumes and the multiple target diastolic volumes, where the first predetermined rule characterizes that the candidate volume corresponding to the minimum value among the multiple candidate volumes is the target systolic volume, and the candidate volume corresponding to the maximum value among the multiple candidate volumes is the target diastolic volume. The obtaining module 840 can be used to perform the operation S240 described above, which will not be elaborated here.

[0187] According to an embodiment of the present invention, the segmentation module 820 includes: a first obtaining sub-module, a second obtaining sub-module, and a third obtaining sub-module.

[0188] The first obtaining sub-module is configured to, for any one of a plurality of frames of echocardiogram images, input the any one of the plurality of frames of echocardiogram images into a semi-supervised depth segmentation model, so as to calculate a depth value for each pixel point by using the semi-supervised depth segmentation model, and obtain a relative depth estimation value corresponding to each pixel point.

[0189] The second obtaining sub-module is configured to screen out, from a plurality of relative depth estimation values, relative depth estimation values corresponding to each pixel point located in the left ventricular region.

[0190] The third obtaining sub-module is configured to perform the above operations on each of the plurality of frames of echocardiogram images, and obtain relative depth estimation values corresponding to each pixel point located in the left ventricular region in each frame of echocardiogram image.

[0191] According to an embodiment of the present invention, the second predetermined rule represents that, when the relative depth estimation value is 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in a region outside the left ventricle, and when the relative depth estimation value is not 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in a region inside the left ventricle.

[0192] According to an embodiment of the present invention, the calculation module 830 includes: a first calculation sub-module.

[0193] The first calculation sub-module is configured to perform a summation process on the relative depth estimation values of each pixel point in each frame of echocardiogram image, and calculate a candidate volume of the left ventricle in each frame of echocardiogram image.

[0194] According to an embodiment of the present invention, the obtaining module 840 includes: a first generating sub-module, a first peak seeking sub-module, and a second calculation sub-module.

[0195] The first generating sub-module is configured to generate an initial volume curve graph of the left ventricle according to a plurality of candidate volumes.

[0196] The first peak seeking sub-module is configured to perform a peak seeking process on the initial volume curve graph by using a peak seeking function, and determine a plurality of target systolic volumes and a plurality of target diastolic volumes.

[0197] The second calculation sub-module is configured to calculate a target ejection function evaluation value according to the plurality of target systolic volumes and the plurality of target diastolic volumes.

[0198] According to an embodiment of the present invention, the second calculation sub-module includes: a first dividing unit, a first calculation unit, and a first obtaining unit.

[0199] A first division unit is configured to perform a cardiac cycle division process on an initial volume curve graph based on a cardiac cycle division rule, so as to obtain a target volume curve graph, where the target volume curve graph represents a graph divided into multiple cardiac cycles, and the cardiac cycle division rule represents that an adjacent target systolic volume and a target diastolic volume are used as one cardiac cycle for division.

[0200] A first calculation unit is configured to perform an ejection calculation on the target systolic volume and the target diastolic volume within each cardiac cycle, so as to obtain an ejection fraction corresponding to each cardiac cycle.

[0201] A first obtaining unit is configured to perform an averaging process on multiple ejection fractions, so as to obtain a target ejection function evaluation value.

[0202] According to an embodiment of the present invention, the apparatus for evaluating an ejection function value based on a semi-supervised deep segmentation model further includes: a training module.

[0203] The training module is configured to train the semi-supervised deep segmentation model.

[0204] According to an embodiment of the present invention, the training module includes: a first acquisition sub-module, a first pre-segmentation sub-module, a first segmentation sub-module, a fourth obtaining sub-module, and a first adjustment sub-module.

[0205] The first acquisition sub-module is configured to acquire a training data set, where the training data set includes an echocardiogram sample video, the echocardiogram sample video is composed of multiple frames of echocardiogram sample images, and among the multiple frames of echocardiogram sample images, there are diastolic sample images with segmentation labels, systolic sample images with segmentation labels, and echocardiogram sample images without segmentation labels.

[0206] The first pre-segmentation sub-module is configured to perform a pre-segmentation process on the echocardiogram sample video, so as to obtain a pre-depth segmentation image corresponding to each frame of echocardiogram sample image in the echocardiogram sample video, where the pre-depth segmentation image includes a pre-depth segmentation label corresponding to each pixel point, and the pre-depth segmentation label represents the depth of each pixel point in the left ventricular region.

[0207] The first segmentation sub-module is configured to input the echocardiogram sample video into the semi-supervised deep segmentation model to be trained for segmentation processing, so as to obtain a segmentation training image corresponding to each frame of echocardiogram sample image in the echocardiogram sample video, where the semi-supervised deep segmentation model to be trained is composed of a first image segmentation network and a second image segmentation network with the same initial weights, and the segmentation training image includes a diastolic training image, a systolic training image, and multiple frames of cardiac cycle training images.

[0208] The fourth obtaining sub-module is configured to obtain a training loss value based on a loss function according to multiple frames of pre-depth segmentation images, diastolic training images, systolic training images, and multiple frames of cardiac cycle training images.

[0209] The first adjustment sub-module is configured to obtain a trained semi-supervised depth segmentation model by adjusting the initial weights of the first image segmentation network and the initial weights of the second image segmentation network according to the training loss value.

[0210] According to an embodiment of the present invention, the fourth obtaining sub-module includes: a second obtaining unit, a third obtaining unit, and a first determining unit.

[0211] The second obtaining unit is configured to obtain a diastolic loss function for each diastolic pixel point in the first diastolic training image according to the training depth of the diastolic pixel point and the pre-depth segmentation label of the first target pixel point, where the first target pixel point is determined from the first pre-depth segmentation image corresponding to the diastolic training image according to the coordinate information of the diastolic pixel point.

[0212] The third obtaining unit is configured to obtain a systolic loss function for each systolic pixel point in the second systolic training image according to the training depth of the systolic pixel point and the pre-depth segmentation label of the second target pixel point, where the second target pixel point is determined from the second pre-depth segmentation image corresponding to the systolic training image according to the coordinate information of the systolic pixel point.

[0213] The first determining unit is configured to determine the training loss value according to the supervised loss value determined based on multiple diastolic loss functions, multiple systolic loss functions, the size information of the diastolic training image, and the size information of the systolic training image, and the target cross pseudo-supervised loss value determined based on the size information of multiple cardiac motion training images, the diastolic training image, the systolic training image, and multiple pre-depth segmentation images.

[0214] According to an embodiment of the present invention, the first determining unit includes: a first obtaining sub-unit, a second obtaining sub-unit, a third obtaining sub-unit, a first determining sub-unit, a second determining sub-unit, and a third determining sub-unit.

[0215] The first obtaining sub-unit is configured to determine a first mean square error function for each first cardiac motion pixel point in multiple first network training images according to the training depth of each first cardiac motion pixel point and the training depth of the third target pixel point, where the third target pixel point is determined from the second network training image according to the coordinate information of the first cardiac motion pixel point.

[0216] A second obtaining subunit, configured to determine, according to the training depth of each second cardiac pixel point and the training depth of a fourth target pixel point, the second mean square error function of each of the multiple second cardiac pixel points in multiple second network training images, where the fourth target pixel point is determined from a first network training image according to the coordinate information of the second cardiac pixel point.

[0217] A third obtaining subunit, configured to process a diastolic training image and a systolic training image to obtain the third mean square error function of each of the multiple diastolic pixel points in a first diastolic training image, the fourth mean square error function of each of the multiple diastolic pixel points in a second diastolic training image, the fifth mean square error function of each of the multiple systolic pixel points in a first systolic training image, and the sixth mean square error function of each of the multiple systolic pixel points in a second systolic training image.

[0218] A first determining subunit, configured to determine a first cross pseudo-supervision loss value based on multiple first mean square error functions, multiple second mean square error functions, and the size information of multiple frames of cardiac training images.

[0219] A second determining subunit, configured to determine a second cross pseudo-supervision loss value based on multiple third mean square error functions, multiple fourth mean square error functions, multiple fifth mean square error functions, multiple sixth mean square error functions, the size information of the diastolic training image, and the size information of the systolic training image.

[0220] A third determining subunit, configured to determine a target cross pseudo-supervision loss value according to the first cross pseudo-supervision loss value and the second cross pseudo-supervision loss value.

[0221] According to an embodiment of the present invention, any plurality of modules among the acquisition module 810, the segmentation module 820, the calculation module 830, and the obtaining module 840 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 810, the segmentation module 820, the calculation module 830, and the obtaining module 840 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or may be implemented by any other reasonable means such as hardware or firmware for integrating or packaging circuits, or may be implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the acquisition module 810, the segmentation module 820, the calculation module 830, and the obtaining module 840 may be at least partially implemented as a computer program module, and when the computer program module is run, corresponding functions may be executed.

[0222] Figure 9 A block diagram of an electronic device for a method of evaluating an ejection function value of a semi-supervised depth segmentation model according to an embodiment of the present invention is shown.

[0223] As Figure 9 shown, the electronic device according to an embodiment of the present invention includes a processor 901, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application specific integrated circuit (ASIC)), and so on. The processor 901 may also include on-board memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0224] In the RAM 903, various programs and data required for the operation of the electronic device are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The processor 901 performs various operations of the method flow according to the embodiments of the present invention by executing the programs in the ROM 902 and / or the RAM 903. It should be noted that the programs may also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 may also perform various operations of the method flow according to the embodiments of the present invention by executing the programs stored in the one or more memories.

[0225] According to an embodiment of the present invention, the electronic device may further include an input / output (I / O) interface 905, and the input / output (I / O) interface 905 is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read from it can be installed into the storage portion 908 as needed.

[0226] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present invention is implemented.

[0227] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the above-described ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903.

[0228] An embodiment of the present invention also includes a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to enable the computer system to implement the method for evaluating the ejection function value based on the semi-supervised depth segmentation model provided by the embodiments of the present invention.

[0229] When the computer program is executed by the processor 901, it executes the above functions defined in the system / apparatus of the embodiments of the present invention. According to an embodiment of the present invention, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0230] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 909, and / or be installed from the removable medium 911. The program code included in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0231] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or be installed from the removable medium 911. When the computer program is executed by the processor 901, it executes the above functions defined in the system of the embodiments of the present invention. According to an embodiment of the present invention, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0232] Those skilled in the art can understand that the features described in the various embodiments of the present invention can be combined or / and combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

Claims

1. A method for evaluating ejection function based on a semi-supervised deep segmentation model, characterized in that: include: Acquiring an ultrasonic cardiology video, wherein the ultrasonic cardiology video includes multiple frames of ultrasonic cardiology images; The ultrasonic video is segmented using a semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel located in the left ventricular region in each frame of the ultrasonic image, wherein the semi-supervised deep segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the depth of each pixel located in the left ventricular region. The training method of the semi-supervised deep segmentation model includes: pre-segmenting the ultrasonic sample image to determine the long axis of the left ventricle, selecting a plurality of cross sections perpendicular to the long axis according to the long axis, and selecting a plurality of cross sections perpendicular to the long axis when the radius from the pixel to the center of the cross section is less than or equal to the long axis. When the length of the cross section is determined, it is determined that the pixel point is located in the left ventricle area, and the depth value of the pixel point is calculated according to the radius from the pixel point to the center of the cross section and the length of the cross section; a first diastolic training image is obtained by inputting the pre-depth segmentation image and the diastolic sample image into the first image segmentation network of the initial model, and a second systolic training image is obtained by inputting the systolic sample image into the second image segmentation network of the initial model, and a supervision loss value is determined; a target cross-pseudo-supervision loss value is obtained according to the diastolic training image, the systolic training image and the multi-frame cardiac training image, and the initial model is trained according to the supervision loss value and the target cross-pseudo-supervision loss value; Calculating a candidate volume of the left ventricle in each frame of the ultrasound cardiogram according to a plurality of relative depth estimation values ​​in each frame of the ultrasound cardiogram; Based on a first predetermined rule, a plurality of target systolic volumes and a plurality of target diastolic volumes are determined from a plurality of candidate volumes, and a target ejection function evaluation value of the left ventricle is obtained according to the plurality of target systolic volumes and the plurality of target diastolic volumes, wherein the first predetermined rule characterizes that a candidate volume corresponding to a minimum value among the plurality of candidate volumes is the target systolic volume, and a candidate volume corresponding to a maximum value among the plurality of candidate volumes is the target diastolic volume.

2. The method according to claim 1, characterized in that The method of using a semi-supervised deep segmentation model to segment the ultrasound video to obtain a relative depth estimation value corresponding to each pixel located in the left ventricle region in each frame of the ultrasound image includes: For any frame of the multiple frames of ultrasound cardiology images, input the any frame of ultrasound cardiology images into the semi-supervised depth segmentation model, so as to calculate the depth value of each pixel point by using the semi-supervised depth segmentation model to obtain a relative depth estimation value corresponding to each pixel point; According to a second predetermined rule, a relative depth estimation value corresponding to each pixel point located in the left ventricle region is obtained by screening from a plurality of relative depth estimation values; The above operation is performed on each frame of the multiple frames of ultrasonic cardiology images to obtain a relative depth estimation value corresponding to each pixel point located in the left ventricle region in each frame of the ultrasonic cardiology images.

3. The method according to claim 2, characterized in that The second predetermined rule represents that when the relative depth estimation value is 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in the area outside the left ventricle; when the relative depth estimation value is not 0, it is confirmed that the pixel point corresponding to the relative depth estimation value is located in the area inside the left ventricle.

4. The method according to claim 1, characterized in that The step of calculating the candidate volume of the left ventricle in each frame of the ultrasound cardiogram according to the plurality of relative depth estimation values ​​in each frame of the ultrasound cardiogram comprises: The relative depth estimation value of each pixel in each frame of the ultrasonic cardiogram is summed up to obtain the candidate volume of the left ventricle in each frame of the ultrasonic cardiogram.

5. The method according to claim 1, characterized in that The step of determining a plurality of target systolic volumes and a plurality of target diastolic volumes from a plurality of candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes, comprises: generating an initial volume curve map of the left ventricle according to the plurality of candidate volumes; Performing peak-finding processing on the initial volume curve graph by using a peak-finding function to determine the multiple target contraction volumes and the multiple target diastolic volumes; The target ejection function evaluation value is calculated based on the multiple target systolic volumes and the multiple target diastolic volumes.

6. The method according to claim 5, characterized in that The step of calculating the target ejection function evaluation value according to the multiple target systolic volumes and the multiple target diastolic volumes includes: Based on a cardiac cycle division rule, the initial volume curve graph is processed by cardiac cycle division to obtain a target volume curve graph, wherein the target volume curve graph represents a curve graph including multiple cardiac cycles, and the cardiac cycle division rule represents division of an adjacent target systolic volume and a target diastolic volume as one cardiac cycle; Performing ejection calculation on the target systolic volume and the target diastolic volume in each cardiac cycle to obtain an ejection fraction corresponding to each cardiac cycle; A plurality of ejection fractions are averaged to obtain the target ejection function assessment value.

7. The method according to claim 1, characterized in that The semi-supervised deep segmentation model is trained using the following methods, including: Acquire a training data set, wherein the training data set includes an ultrasound cardiology sample video, the ultrasound cardiology sample video is composed of multiple frames of ultrasound cardiology sample images, and the multiple frames of ultrasound cardiology sample images include diastolic sample images containing segmentation labels, contraction sample images containing segmentation labels, and ultrasound cardiology sample images without segmentation labels; Performing pre-segmentation processing on the ultrasonic cardiology sample video to obtain a pre-depth segmentation image corresponding to each frame of the ultrasonic cardiology sample image in the ultrasonic cardiology sample video, wherein the pre-depth segmentation image includes a pre-depth segmentation label corresponding to each pixel point, and the pre-depth segmentation label represents the depth of each pixel point in the left ventricle region; Inputting the ultrasound cardiology sample video into the semi-supervised deep segmentation model to be trained for segmentation processing, and obtaining a segmentation training image corresponding to each frame of ultrasound sample image in the ultrasound cardiology sample video, wherein the semi-supervised deep segmentation model to be trained is composed of a first image segmentation network and a second image segmentation network with the same initial weights, and the segmentation training images include a diastolic training image, a systolic training image, and multiple frames of cardiac training images; Based on the loss function, a training loss value is obtained according to the multiple frames of the pre-depth segmentation images, the diastolic training images, the systolic training images and the multiple frames of cardiac training images; According to the training loss value, the trained semi-supervised deep segmentation model is obtained by adjusting the initial weights of the first image segmentation network and the initial weights of the second image segmentation network.

8. The method according to claim 7, characterized in that The diastolic training image includes a first diastolic training image processed by the first image segmentation network and a second diastolic training image processed by the second image segmentation network, the contraction training image includes a first contraction training image processed by the first image segmentation network and a second contraction training image processed by the second image segmentation network, the first diastolic training image and the second diastolic training image include diastolic pixels, and the first contraction training image and the second contraction training image include contraction pixels; The method of obtaining a training loss value based on the loss function according to the multiple frames of the pre-depth segmentation images, the diastolic training images, the systolic training images and the multiple frames of cardiac training images comprises: For each of the diastolic pixel points in the first diastolic training image, obtaining respective diastolic loss functions of the plurality of diastolic pixel points in the first diastolic training image according to the training depth of the diastolic pixel point and the pre-depth segmentation label of the first target pixel point, wherein the first target pixel point is determined from the first pre-depth segmentation image corresponding to the diastolic training image according to the coordinate information of the diastolic pixel point; For each of the shrunken pixel points in the second shrunken training image, obtaining a respective shrunken loss function of a plurality of the shrunken pixel points in the second shrunken training image according to the training depth of the shrunken pixel point and the pre-depth segmentation label of the second target pixel point, wherein the second target pixel point is determined from a second pre-depth segmentation image corresponding to the shrunken training image according to the coordinate information of the shrunken pixel point; The training loss value is determined according to the supervised loss value determined based on the multiple diastolic loss functions, the multiple systolic loss functions, the size information of the diastolic training image and the size information of the systolic training image, and the target cross-pseudo-supervised loss value determined based on the size information of the multiple-frame cardiac training images, the diastolic training images, the systolic training images and the multiple-frame cardiac training images.

9. The method according to claim 8, characterized in that The heartbeat training image includes a first network training image processed by the first image segmentation network and a second network training image processed by the second image segmentation network, the first network training image includes first heartbeat pixels, and the second network training image includes second heartbeat pixels; The target cross pseudo-supervision loss value is determined as follows: Determine, according to the training depth of each of the first cardiac pixel points and the training depth of the third target pixel point, a first mean square error function of each of the first cardiac pixel points in the first network training images, wherein the third target pixel point is determined from the second network training image according to the coordinate information of the first cardiac pixel point; Determine, according to the training depth of each of the second cardiac pixel points and the training depth of the fourth target pixel point, a second mean square error function of each of the second cardiac pixel points in the second network training images, wherein the fourth target pixel point is determined from the first network training image according to the coordinate information of the second cardiac pixel point; Processing the diastolic training image and the systolic training image to obtain a third mean square error function of each of the diastolic pixels in the first diastolic training image, a fourth mean square error function of each of the diastolic pixels in the second diastolic training image, a fifth mean square error function of each of the systolic pixels in the first systolic training image, and a sixth mean square error function of each of the systolic pixels in the second systolic training image; Determine a first cross pseudo-supervision loss value based on the multiple first mean square error functions, the multiple second mean square error functions and size information of the multiple frames of cardiac training images; Determining a second cross pseudo-supervision loss value based on the plurality of third mean square error functions, the plurality of fourth mean square error functions, the plurality of fifth mean square error functions, the plurality of sixth mean square error functions, the size information of the relaxation training image, and the size information of the contraction training image; A target cross-pseudo-supervised loss value is determined according to the first cross-pseudo-supervised loss value and the second cross-pseudo-supervised loss value.

10. A device for evaluating ejection function based on a semi-supervised deep segmentation model, characterized in that: include: An acquisition module, used for acquiring an ultrasonic cardiology video, wherein the ultrasonic cardiology video includes multiple frames of ultrasonic cardiology images; A segmentation module is used to segment the ultrasound video using a semi-supervised deep segmentation model to obtain a relative depth estimation value corresponding to each pixel located in the left ventricular region in each frame of the ultrasound image, wherein the semi-supervised deep segmentation model includes a first image segmentation network and a second image segmentation network with different weights, and the relative depth estimation value represents the depth of each pixel located in the left ventricular region. The training method of the semi-supervised deep segmentation model includes: pre-segmenting the ultrasound sample image to determine the long axis of the left ventricle, selecting a plurality of cross sections perpendicular to the long axis according to the long axis, and selecting a plurality of cross sections when the radius from the pixel to the center of the cross section is less than or equal to When the length of the cross section is greater than or equal to that of the cross section, it is determined that the pixel point is located in the left ventricle region, and the depth value of the pixel point is calculated according to the radius from the pixel point to the center of the cross section and the length of the cross section; a first diastolic training image is obtained by inputting a pre-depth segmentation image and a diastolic sample image into a first image segmentation network of an initial model, and a second systolic training image is obtained by inputting a systolic sample image into a second image segmentation network of the initial model, and a supervised loss value is obtained; a target cross-pseudo-supervised loss value is obtained according to the diastolic training image, the systolic training image and the multi-frame cardiac training images, and the initial model is trained according to the supervised loss value and the target cross-pseudo-supervised loss value; A calculation module, configured to calculate a candidate volume of the left ventricle in each frame of the ultrasonic cardiogram according to a plurality of relative depth estimation values ​​in each frame of the ultrasonic cardiogram; an obtaining module, for determining a plurality of target systolic volumes and a plurality of target diastolic volumes from a plurality of candidate volumes based on a first predetermined rule, and obtaining a target ejection function evaluation value of the left ventricle according to the plurality of target systolic volumes and the plurality of target diastolic volumes, wherein the first predetermined rule indicates that the candidate volume corresponding to the minimum value among the plurality of candidate volumes is the target systolic volume, and the candidate volume corresponding to the maximum value among the plurality of candidate volumes is the target diastolic volume.

Images