Echocardiographic acoustic window scan guidance method and ultrasound imaging system

Abstract

The application provides an echocardiogram acoustic window scanning guidance method, an ultrasonic imaging system, an electronic device and a readable storage medium. The method comprises the following steps: acquiring a current frame echocardiogram image collected by an ultrasonic probe to be guided at a target acoustic window; identifying the acquired current frame echocardiogram image by using a digital heart plane identification model to acquire digital heart plane representation information of a section corresponding to the current frame echocardiogram image in a space of a pre-established digital three-dimensional heart model; and acquiring target motion information of the ultrasonic probe to be guided according to the digital heart plane representation information of the current frame echocardiogram image and digital heart plane representation information of a target standard section in the space of the digital three-dimensional heart model, so as to guide the ultrasonic probe to be guided to move to a direction corresponding to the target standard section. The application can enable non-ultrasonic professionals to easily perform echocardiogram acoustic window scanning, and reduce the work burden of senior ultrasonic doctors.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an echocardiography acoustic window scanning guided method, an ultrasound imaging system, an electronic device, and a readable storage medium. Background Technology

[0002] Methods for examining suspicious lesions in the heart include medical imaging techniques such as CT, SPECT, PET, MRI, and ultrasound. However, because the heart is a constantly moving organ, these modalities can produce motion artifacts in cardiac imaging, affecting the clinician's diagnosis. Ultrasound imaging utilizes the reception and processing of signals reflected from ultrasound waves within the body to obtain images of regions of interest. Echocardiography is a common method for examining the heart using ultrasound. Its imaging characterizes the degree of ultrasound reflection by cardiac tissue, allowing real-time examination of the velocity and amplitude of blood and myocardial tissue movement at any location using continuous ultrasound waves. It is characterized by its real-time nature, safety, low cost, and convenience. However, because echocardiography examinations largely depend on the operator's scanning technique, obtaining standard sections requires considerable clinical experience. In settings such as primary hospitals, emergency rooms, and ambulances, a method that guides non-ultrasound professionals to easily perform echocardiographic scans is highly valuable.

[0003] It should be noted that the information disclosed in the background section of this invention is intended only to enhance the understanding of the general background of this invention, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a method for guiding echocardiographic acoustic window scanning, an ultrasound imaging system, an electronic device, and a readable storage medium, which can guide echocardiographic acoustic window scanning for non-ultrasound professionals without introducing additional auxiliary devices.

[0005] To achieve the above objectives, the present invention provides a method for guiding echocardiography with acoustic window scanning, comprising:

[0006] Acquire the current frame of echocardiogram image captured by the ultrasound probe to be guided within the target acoustic window;

[0007] A pre-trained digital cardiac plane recognition model is used to identify the current frame echocardiogram to obtain the digital cardiac plane representation information of the cross-section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model.

[0008] Based on the digital cardiac plane representation information corresponding to the current frame echocardiogram and the digital cardiac plane representation information of the target standard section under the target acoustic window in the space of the digital three-dimensional heart model, the target motion information of the ultrasound probe to be guided is obtained so as to guide the ultrasound probe to move to the direction corresponding to the target standard section.

[0009] Optionally, the digital heart plane recognition model is trained through the following steps:

[0010] Multiple first training samples are obtained, wherein each first training sample includes an echocardiogram training image and its corresponding first label. The first label represents the digital cardiac plane representation information of the cross section corresponding to the echocardiogram training image in the space where the digital three-dimensional heart model is located. All the echocardiogram training images are acquired under the target acoustic window, and for each standard cross section under the target acoustic window, at least one cross section corresponding to the echocardiogram training image is the standard cross section.

[0011] The pre-built digital heart plane recognition model is trained based on the first training sample until a first preset training termination condition is met. The first preset training termination condition is that the number of training iterations reaches a preset first iteration number or the corresponding first loss function value is less than a first threshold.

[0012] Optionally, obtaining multiple first training samples includes:

[0013] An ultrasound probe with a fixed inertial measurement unit is used to continuously scan the target acoustic window to obtain echocardiographic training images of various standard and non-standard sections under the target acoustic window, as well as the attitude information of the ultrasound probe when acquiring each of the echocardiographic training images, wherein the attitude information is acquired by the inertial measurement unit.

[0014] For each of the aforementioned echocardiographic training images, based on the posture information corresponding to the echocardiographic training image and the pre-acquired digital cardiac plane regression model, the digital cardiac plane representation information of the cross section corresponding to the echocardiographic training image in the space where the digital three-dimensional heart model is located is obtained, and the echocardiographic training image and its corresponding digital cardiac plane representation information are combined to form an example of the first training sample.

[0015] Optionally, the digital cardiac plane regression model is a pre-trained neural network regression model. The step of obtaining the digital cardiac plane representation information of the cross-section corresponding to the echocardiographic training image in the space of the digital three-dimensional cardiac model, based on the pose information corresponding to the echocardiographic training image and the pre-acquired digital cardiac plane regression model, includes:

[0016] The corresponding rotational affine matrix is ​​obtained based on the posture information corresponding to the echocardiogram training image, and the rotational affine matrix is ​​converted into the corresponding rotational affine vector.

[0017] The affine rotation vector is input into the digital heart-plane regression model to obtain the corresponding prediction vector;

[0018] The prediction vector is converted into a prediction matrix corresponding to the echocardiogram training image, and the prediction matrix corresponding to the echocardiogram training image is orthogonalized to obtain the orthogonal matrix corresponding to the echocardiogram training image, thereby obtaining the digital cardiac plane representation information of the cross-section corresponding to the echocardiogram training image in the space where the digital three-dimensional heart model is located.

[0019] Optionally, the digital cardiac plane regression model is trained through the following steps:

[0020] The echocardiogram training image with the cross-section of the standard cross-section under the target acoustic window is selected as the target echocardiogram training image;

[0021] For each target echocardiogram training image, the digital cardiac plane representation information of the standard section corresponding to the target echocardiogram training image in the space where the digital three-dimensional heart model is located is used as a second label and the pose information corresponding to the target echocardiogram training image to form a second training sample.

[0022] The pre-built neural network regression model is trained based on the second training sample until the second preset training termination condition is met, so as to obtain the digital heart plane regression model. The second preset training termination condition is that the number of training iterations reaches the preset second iteration number or the corresponding second loss function value is less than the second threshold.

[0023] Optionally, the orthogonalization process performed on the prediction matrix corresponding to the echocardiographic training image to obtain the orthogonal matrix corresponding to the echocardiographic training image includes:

[0024] Singular value decomposition is performed on the prediction matrix corresponding to the echocardiogram training image, and the orthogonal matrix corresponding to the echocardiogram training image is obtained based on the singular value decomposition result of the prediction matrix corresponding to the echocardiogram training image.

[0025] Optionally, the step of using a pre-trained digital cardiac plane recognition model to recognize the acquired current frame echocardiogram to obtain digital cardiac plane representation information of the cross-section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model includes:

[0026] A pre-trained digital cardiac plane recognition model is used to identify the current frame of echocardiogram to obtain the prediction matrix corresponding to the current frame of echocardiogram.

[0027] The prediction matrix corresponding to the current frame echocardiogram is orthogonalized to obtain the digital cardiac plane representation information of the cross-section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model.

[0028] Optionally, the method further includes:

[0029] Based on the digital heart plane representation information of all standard sections under the target acoustic window in the space of the digital three-dimensional heart model obtained in advance, the position information of the intersection points of all standard sections under the target acoustic window is obtained.

[0030] Based on the position information of the intersection points of all standard sections under the target acoustic window, the position information of the ultrasound probe to be guided in the space of the digital three-dimensional heart model is determined;

[0031] Based on the position information of the ultrasound probe to be guided in the space of the digital three-dimensional heart model, the digital cardiac plane corresponding to the current frame of echocardiogram and the digital cardiac plane corresponding to the target standard section are translated to the position of the ultrasound probe to be guided in the space of the digital three-dimensional heart model.

[0032] To achieve the above objectives, the present invention also provides an electronic device, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the echocardiography acoustic window scanning guided method described above.

[0033] To achieve the above objectives, the present invention also provides an ultrasound imaging system, including an ultrasound probe and a controller connected in communication, wherein the ultrasound probe is configured to acquire echocardiogram images and transmit them to the controller, and the controller is configured to implement the echocardiogram acoustic window scanning guided method described above.

[0034] Optionally, the ultrasound imaging system further includes a display communicatively connected to the controller, the display being configured to display at least one of the echocardiogram, the digital three-dimensional heart model, the digital cardiac plane corresponding to the target standard section, and the digital cardiac plane corresponding to the current frame echocardiogram.

[0035] To achieve the above objectives, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the echocardiography acoustic window scanning guided method described above.

[0036] Compared with the prior art, the echocardiography acoustic window scanning guided method, ultrasound imaging system, electronic device, and readable storage medium provided by the present invention have the following advantages:

[0037] The echocardiography acoustic window scanning guidance method provided by the present invention first acquires the current frame echocardiogram image captured by the ultrasound probe to be guided in the target acoustic window; then, it uses a pre-trained digital cardiac plane recognition model to recognize the acquired current frame echocardiogram image to obtain the digital cardiac plane representation information of the section corresponding to the current frame echocardiogram image in the space of the pre-established digital three-dimensional heart model; finally, based on the digital cardiac plane representation information corresponding to the current frame echocardiogram image and the pre-acquired digital cardiac plane representation information of the target standard section under the target acoustic window in the space of the digital three-dimensional heart model, it obtains the target motion information of the ultrasound probe to be guided, so as to guide the ultrasound probe to be guided to move to the direction corresponding to the target standard section. Therefore, the echocardiographic acoustic window scanning guidance method provided by this invention introduces a digital three-dimensional heart model. Using the digital cardiac plane representation information corresponding to the target standard section and the digital cardiac plane representation information corresponding to the current section (i.e., the section corresponding to the current frame of echocardiographic image) as references, it provides guidance for the movement of the ultrasound probe during echocardiographic acoustic window (e.g., the apical acoustic window) scanning. This allows non-ultrasound professionals to easily perform echocardiographic acoustic window (e.g., the apical acoustic window) scanning, thereby reducing the workload of senior ultrasound physicians. Furthermore, this invention completes the ultrasound probe movement guidance without introducing additional auxiliary devices or modifying the ultrasound probe, further reducing the operational difficulty for ultrasound scanners and saving costs.

[0038] Since the ultrasound imaging system, electronic device, and readable storage medium provided by this invention belong to the same inventive concept as the echocardiography acoustic window scanning guided method provided by this invention, the ultrasound imaging system, electronic device, and readable storage medium provided by this invention have all the advantages of the echocardiography acoustic window scanning guided method provided by this invention. For details, please refer to the relevant description above. Therefore, the beneficial effects of the ultrasound imaging system, electronic device, and readable storage medium provided by this invention will not be elaborated here. Attached Figure Description

[0039] Figure 1 A flowchart of an echocardiography window scanning guided method provided in one embodiment of the present invention;

[0040] Figure 2 A flowchart illustrating the training process of a digital heart plane recognition model provided in one embodiment of the present invention;

[0041] Figure 3 A flowchart for obtaining a first training sample provided in one embodiment of the present invention;

[0042] Figure 4 A flowchart illustrating the training process of a digital cardiac plane regression model provided in one embodiment of the present invention;

[0043] Figure 5 This is a block diagram of an ultrasound imaging system provided in one embodiment of the present invention;

[0044] Figure 6 This is a block diagram of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0045] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the echocardiography acoustic window scanning guided method, electronic device, and readable storage medium proposed in this invention. The advantages and features of the invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the embodiments of the invention. Please refer to the drawings to make the objectives, features, and advantages of the invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the implementation conditions of the invention. Any modifications to the structure, changes in proportions, or adjustments to the size, provided that the effects and objectives achieved by the invention are the same or similar, should still fall within the scope of the technical content disclosed in the invention. Specific design features of the invention disclosed herein, including, for example, specific dimensions, orientations, positions, and shapes, will be determined in part by the specific application and usage environment. Furthermore, in the embodiments described below, the same reference numerals are sometimes used across different drawings to denote the same parts or parts having the same function, omitting repeated descriptions. In this specification, similar reference numerals and letters are used to denote similar items; therefore, once an item is defined in one figure, it need not be discussed further in subsequent figures. Furthermore, if the methods described herein involve a series of steps, and the order of these steps presented herein is not necessarily the only possible order in which they can be performed, some of the described steps may be omitted and / or other steps not described herein may be added to the method.

[0046] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The singular forms “a,” “an,” and “the” include plural objects. The term “or” is generally used to mean “and / or,” the term “several” is generally used to mean “at least one,” and the term “at least two” is generally used to mean “two or more.” Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0047] Furthermore, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0048] The core idea of ​​this invention is to provide a method, electronic device, and readable storage medium for guiding echocardiographic acoustic window scanning, which can guide echocardiographic acoustic window scanning for non-ultrasound professionals without introducing additional auxiliary devices. It should be noted that the echocardiographic acoustic window scanning guidance method provided by this invention can be applied to the electronic device provided by this invention. The electronic device provided by this invention can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a mobile phone, tablet computer, or other hardware device with various operating systems. It should also be noted that, as those skilled in the art will understand, this invention can guide echocardiographic scans not only under the apical acoustic window, but also under other acoustic windows such as the suprasternal notch acoustic window, the left parasternal acoustic window, and the subxiphoid acoustic window. The standard sections under the apical acoustic window include the four-chamber standard section, the two-chamber standard section, and the three-chamber standard section; the standard sections under the left parasternal acoustic window include the parasternal left ventricular long axis standard section (PLAX standard section), the parasternal short axis standard section (PSAX standard section), and the papillary muscle short axis standard section; the standard sections under the subxiphoid acoustic window include the subxiphoid four-chamber standard section (S4C standard section); and the standard sections under the suprasternal notch acoustic window include the suprasternal notch aortic arch long axis standard section.

[0049] To achieve the above-mentioned goals, this invention provides a method for guiding echocardiographic window scanning. Please refer to [the relevant documentation]. Figure 1 The diagram illustrates a flowchart of an echocardiographic window scanning guided method according to an embodiment of the present invention. Figure 1 As shown, the echocardiography window scanning guided method provided by the present invention includes the following steps:

[0050] Step S100: Acquire the current frame echocardiogram image captured by the ultrasound probe to be guided in the target acoustic window.

[0051] Step S200: Use a pre-trained digital cardiac plane recognition model to recognize the acquired current frame echocardiogram to obtain the digital cardiac plane representation information of the cross-section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model.

[0052] Step S300: Based on the digital cardiac plane representation information corresponding to the current frame echocardiogram and the digital cardiac plane representation information of the target standard section under the target acoustic window in the space where the digital three-dimensional heart model is located, obtain the target motion information of the ultrasound probe to be guided, so as to guide the ultrasound probe to be guided to move to the direction corresponding to the target standard section.

[0053] Therefore, the echocardiographic acoustic window scanning guidance method provided by this invention introduces a digital three-dimensional heart model. Using the digital cardiac plane representation information corresponding to the target standard section and the digital cardiac plane representation information corresponding to the current section (i.e., the section corresponding to the current frame of echocardiographic image) as references, it provides guidance for the movement of the ultrasound probe during echocardiographic acoustic window (e.g., the apical acoustic window) scanning. This allows non-ultrasound professionals to easily perform echocardiographic acoustic window (e.g., the apical acoustic window) scanning, thereby reducing the workload of senior ultrasound physicians. Furthermore, this invention completes the ultrasound probe movement guidance without introducing additional auxiliary devices or modifying the ultrasound probe, further reducing the operational difficulty for ultrasound scanners and saving costs. It should be noted that, as those skilled in the art will understand, a digital heart refers to a digital three-dimensional heart model created using image processing technology combined with other technologies such as artificial intelligence to mimic the human heart, and a digital cardiac plane refers to the plane containing the section on the digital three-dimensional heart model (i.e., the plane containing the section of the digital heart).

[0054] Specifically, relying on the clinical experience of senior ultrasound physicians, the planar representations of each standard section under the target acoustic window in the space of the digital three-dimensional heart model can be obtained, thereby acquiring the digital cardiac plane representation information corresponding to each standard section under the target acoustic window. Taking the apical acoustic window as the target acoustic window as an example, relying on the clinical experience of senior ultrasound physicians, the digital cardiac plane representation information corresponding to the four-chamber standard section, the three-chamber standard section, and the two-chamber standard section can be obtained. Furthermore, based on prior knowledge, scanning of the echocardiographic acoustic window (e.g., scanning of the apical acoustic window) only requires tilting, rotating, and oscillating the ultrasound probe, meaning the position of the ultrasound probe remains unchanged. Therefore, the intersection of the planes can be calculated based on the planar representation of each standard section under the target acoustic window in the space of the digital three-dimensional heart model (i.e., based on the three Euler angles α, β, and γ of the digital heart plane corresponding to each standard section under the target acoustic window in the space of the digital three-dimensional heart model). This intersection can be regarded as the coordinate position of the ultrasound probe in the space of the digital three-dimensional heart model. Taking the apical acoustic window as the target acoustic window as an example, the intersection of the planes can be calculated based on the planar representation of the four-chamber standard section, the three-chamber standard section, and the two-chamber standard section in the space of the digital three-dimensional heart model, thus determining the position coordinates of the ultrasound probe in the space of the digital three-dimensional heart model. Due to constraints, the coordinates of the ultrasound probe remain constant in the space of the digital three-dimensional heart model. Therefore, the digital cardiac plane corresponding to each frame of echocardiogram can be characterized by three Euler angles α, β, and γ. Based on the three Euler angles α, β, and γ of the digital cardiac plane corresponding to each frame, the representation information of the digital cardiac plane corresponding to that frame can be obtained (for example, trigonometric functions related to the three Euler angles α, β, and γ of the digital cardiac plane corresponding to that frame can be used to represent the representation information of the digital cardiac plane corresponding to that frame). It should be noted that, as those skilled in the art will understand, the Euler angle α of the digital cardiac plane corresponding to each frame of echocardiogram corresponds to the tilt angle of the ultrasound probe, the Euler angle β corresponds to the rotation angle of the ultrasound probe, and the Euler angle γ corresponds to the swing angle of the ultrasound probe. It should also be noted that, as those skilled in the art will understand, since at least two of the three movements of tilting, rotating, and swinging are required to change the acquisition plane of the ultrasound probe from one standard plane to another, that is, at least two of the three Euler angles α, β, γ of each standard plane under the target acoustic window in the digital heart plane in the space where the digital three-dimensional heart model is located are different, therefore, each standard plane under the target acoustic window must intersect at a point in the plane in the space where the digital three-dimensional heart model is located.

[0055] Please continue to refer to this. Figure 2 The diagram illustrates the training flowchart of a digital cardiac plane recognition model provided by an embodiment of the present invention. Figure 2 As shown, in one exemplary implementation, the digital heart plane recognition model is trained through the following steps:

[0056] Multiple first training samples are obtained, wherein each first training sample includes an echocardiogram training image and its corresponding first label. The first label represents the digital cardiac plane representation information of the cross section corresponding to the echocardiogram training image in the space where the digital three-dimensional heart model is located. All the echocardiogram training images are acquired under the target acoustic window, and for each standard cross section under the target acoustic window, at least one cross section corresponding to the echocardiogram training image is the standard cross section.

[0057] The pre-built digital heart plane recognition model is trained based on the first training sample until a first preset training termination condition is met. The first preset training termination condition is that the number of training iterations reaches a preset first iteration number or the corresponding first loss function value is less than a first threshold.

[0058] Specifically, taking the apical acoustic window as the target acoustic window as an example, some echocardiographic training images correspond to a four-chamber standard section, some correspond to a three-chamber standard section, and some correspond to a two-chamber standard section. Preferably, the pre-built digital cardiac plane recognition model is trained using gradient descent. Since the model training process is actually the process of minimizing the loss function, and finding the derivative can quickly and easily achieve this goal, this method of finding the derivative is gradient descent. Therefore, using gradient descent to train the digital cardiac plane recognition model can quickly and easily achieve the training of the digital cardiac plane recognition model. It should be noted that, as those skilled in the art will understand, the present invention does not limit the specific network structure of the digital cardiac plane recognition model. The network structure of the digital cardiac plane recognition model can refer to the regression network based on deep learning in the prior art, and will not be elaborated here. It should also be noted that, as those skilled in the art will understand, the first number of iterations and the first threshold can be set according to specific circumstances, and the present invention does not limit them. For example, the first threshold can be set to 0.001.

[0059] Please continue to refer to this. Figure 3 The diagram illustrates a flowchart of obtaining a first training sample according to an embodiment of the present invention. Figure 3 As shown, in one exemplary implementation, obtaining multiple first training samples includes:

[0060] An ultrasound probe with a fixed inertial measurement unit is used to continuously scan the target acoustic window to obtain echocardiographic training images of various standard and non-standard sections under the target acoustic window, as well as the attitude information of the ultrasound probe when acquiring each of the echocardiographic training images, wherein the attitude information is acquired by the inertial measurement unit.

[0061] For each of the aforementioned echocardiographic training images, based on the posture information corresponding to the echocardiographic training image and the pre-acquired digital cardiac plane regression model, the digital cardiac plane representation information of the cross section corresponding to the echocardiographic training image in the space where the digital three-dimensional heart model is located is obtained, and the echocardiographic training image and its corresponding digital cardiac plane representation information are combined to form a first training sample.

[0062] Specifically, taking the apical acoustic window as the target acoustic window as an example, a senior ultrasound physician can be invited to perform an echocardiographic scan of the apical acoustic window to obtain ultrasound video of the transition between different standard sections of the echocardiogram (i.e., obtaining ultrasound video of the transition between the four-chamber standard section, the two-chamber standard section, and the three-chamber standard section of the echocardiogram, which is also obtaining echocardiographic training images of various standard sections and non-standard sections under the target acoustic window). At the same time, the attitude data of the ultrasound probe output by the inertial measurement unit is recorded. The echocardiographic training images acquired by the ultrasound probe can be saved in real time through the acquisition card.

[0063] It should be noted that, as those skilled in the art will understand, the present invention does not limit the specific structure of the inertial measurement unit. The inertial measurement unit can be any element capable of measuring the tilt angle, rotation angle, and swing angle of the ultrasound probe. Furthermore, based on the ultrasound video obtained by senior ultrasound physicians scanning the apical acoustic window of the heart during echocardiography, each frame of the echocardiographic training image in the ultrasound video can be matched one-to-one with the ultrasound probe attitude information output by the inertial measurement unit according to the video frame timestamp.

[0064] In one exemplary embodiment, the digital cardiac plane regression model is a pre-trained neural network regression model. The step of obtaining digital cardiac plane representation information of the cross-section corresponding to the echocardiographic training image in the space of the digital three-dimensional heart model, based on the pose information corresponding to the echocardiographic training image and the pre-acquired digital cardiac plane regression model, includes:

[0065] The first rotational affine matrix is ​​obtained based on the posture information corresponding to the echocardiogram training image, and the first rotational affine matrix is ​​converted into the corresponding first rotational affine vector.

[0066] The first affine rotation vector is input into the digital heart-plane regression model to obtain the corresponding first prediction vector;

[0067] The first prediction vector is converted into the corresponding first prediction matrix, and the first prediction matrix is ​​orthogonalized to obtain the corresponding first orthogonal matrix, thereby obtaining the digital cardiac plane representation information of the cross-section corresponding to the echocardiogram in the space where the digital three-dimensional heart model is located.

[0068] Specifically, assuming the posture information corresponding to the echocardiogram training image is the tilt angle of the ultrasound probe as α', the rotation angle of the ultrasound probe as β', and the swing angle of the ultrasound probe as γ', the first rotational affine matrix R corresponding to the echocardiogram training image can be calculated according to the following formula (1). B :

[0069]

[0070] By using the 3×3 first rotational affine matrix R B Expanding the vector, a 1×9 first affine rotation vector can be obtained. By inputting this 1×9 first affine rotation vector into the digital cardiac plane regression model, a 1×9 first prediction vector can be output. Transforming the 1×9 first prediction vector yields a corresponding 3×3 first prediction matrix. Orthogonalizing the 3×3 first prediction matrix yields a corresponding 3×3 first orthogonal matrix. This first orthogonal matrix represents the digital cardiac plane representation information of the section corresponding to the echocardiographic training image in the space of the digital three-dimensional cardiac model. It should be noted that, as those skilled in the art will understand, assuming the three Euler angles corresponding to the section corresponding to the echocardiographic training image in the digital cardiac plane of the digital three-dimensional cardiac model are α, β, and γ, then the first orthogonal matrix R... D It can be represented as:

[0071]

[0072] Because the representation of Euler angles in Euclidean space suffers from discontinuity, this problem can be solved by converting the posture information (i.e., tilt angle α', rotation angle β', and oscillation angle γ') corresponding to the echocardiographic training image into corresponding trigonometric function values, allowing for continuous representation. It should be noted that, as those skilled in the art will understand, this invention does not limit the specific network structure of the digital cardiac plane regression model; the network structure of the digital cardiac plane regression model can employ any network structure capable of performing the regression task.

[0073] By orthogonalizing the first prediction matrix, an orthogonal matrix that satisfies the properties of a rotation matrix can be obtained. In other words, the three Euler angles α, β, and γ corresponding to the cross-section of the echocardiogram training image in the space of the digital three-dimensional heart model are obtained and satisfy the property of mutual orthogonality.

[0074] It should be noted that, as those skilled in the art will understand, the digital cardiac plane representation information of the target standard section under the target acoustic window in the space where the digital three-dimensional heart model is located can also be represented by a 3×3 matrix similar to the first rotational affine matrix, and will not be elaborated further. It should also be noted that, as those skilled in the art will understand, the digital cardiac plane regression model can also be a linear regression function model used to characterize the mapping relationship between the ultrasound probe's posture and the digital cardiac plane Euler angle information: Y = XA + ε, where Y is the digital cardiac plane Euler angle, X is the ultrasound probe's posture, ε is the residual, and A is the fitting parameter. Based on this linear regression function model, the digital cardiac plane Euler angles corresponding to the echocardiographic training images acquired by the ultrasound probe in various postures can also be obtained, thereby acquiring the corresponding digital cardiac plane representation information.

[0075] In one exemplary embodiment, the orthogonalization of the first prediction matrix to obtain a corresponding first orthogonal matrix includes:

[0076] Perform singular value decomposition on the first prediction matrix, and obtain the corresponding first orthogonal matrix based on the singular value decomposition result of the first prediction matrix.

[0077] Specifically, the first prediction matrix R can be calculated according to the following formula (2). P Perform orthogonalization to obtain the corresponding first orthogonal matrix R. D :

[0078]

[0079] Where SVD represents singular value decomposition, T represents the transpose of a matrix, det represents the determinant of a matrix, and diag represents a diagonal matrix.

[0080] Therefore, by first preparing the first prediction matrix R P Singular value decomposition yields 3×3 orthogonal matrices U and V, as well as a 3×3 diagonal matrix Σ. Σ' is then calculated from U and V. Finally, the inverse operation of singular value decomposition is performed using U, V, and Σ' to obtain the first orthogonal matrix R. D .

[0081] Please continue to refer to this. Figure 4The diagram illustrates the training flowchart of a digital cardiac plane regression model provided by an embodiment of the present invention. Figure 4 As shown, in one exemplary implementation, the digital cardiac plane regression model is trained through the following steps:

[0082] The echocardiogram training image with the cross-section of the standard cross-section under the target acoustic window is selected as the target echocardiogram training image;

[0083] For each target echocardiogram training image, the digital cardiac plane representation information of the standard section corresponding to the target echocardiogram training image in the space where the digital three-dimensional heart model is located is used as a second label and the pose information corresponding to the target echocardiogram training image to form a second training sample.

[0084] The pre-built neural network regression model is trained based on the second training sample until the second preset training termination condition is met, so as to obtain the digital heart plane regression model. The second preset training termination condition is that the number of training iterations reaches the preset second iteration number or the corresponding second loss function value is less than the second threshold.

[0085] Specifically, taking the apical acoustic window as the target acoustic window as an example, several echocardiographic training images conforming to the four-chamber standard section, the three-chamber standard section, and the two-chamber standard section are selected as target echocardiographic training images. Since each second training sample includes the posture information of the ultrasound probe and the corresponding digital cardiac plane representation information, the mapping relationship between the posture information of the ultrasound probe and the digital plane representation information can be fitted by training the pre-built neural network regression model using the second training samples. Preferably, the gradient descent method is used to train the pre-built neural network regression model, thereby improving the training speed and facilitating the rapid and simple training of the digital cardiac plane regression model. It should be noted that, as those skilled in the art will understand, the present invention does not limit the specific network structure of the digital cardiac plane regression model, and the digital cardiac plane regression model can be obtained by using any neural network regression model with any network structure in the prior art. It should also be noted that, as those skilled in the art will understand, the second iteration number and the second threshold can be set according to specific circumstances, and the present invention does not limit them. For example, the second threshold can be set to 0.001.

[0086] In one exemplary embodiment, training the pre-built neural network regression model using the second training samples and gradient descent until a second preset training termination condition is met includes:

[0087] The second rotation affine matrix is ​​obtained based on the posture information corresponding to the target echocardiogram training image of the second training sample, and the second rotation affine matrix is ​​converted into the corresponding second rotation affine vector.

[0088] The second rotational affine vector is input into a pre-built neural network regression model to obtain the corresponding second prediction vector;

[0089] The second prediction vector is converted into the corresponding second prediction matrix, and the second prediction matrix is ​​orthogonalized to obtain the corresponding second orthogonal matrix.

[0090] The second orthogonal matrix is ​​converted into the corresponding third prediction vector, and the second label of the second training sample is converted into the corresponding first digital heart plane representation vector;

[0091] Calculate the second loss function value between the third prediction vector and the first digital heart plane representation vector;

[0092] Determine whether the value of the second loss function is less than the second threshold. If so, end the training.

[0093] Specifically, referring to formula (1), the second rotational affine matrix corresponding to the target echocardiographic training image of the second training sample (a 3×3 matrix related to the posture information corresponding to the target echocardiographic training image) can be obtained; by expanding the 3×3 second rotational affine matrix, a 1×9 second rotational affine vector can be obtained; by inputting the 1×9 second rotational affine vector into the neural network regression model, a 1×9 second prediction vector can be output; by transforming the 1×9 second prediction vector, a corresponding 3×3 second prediction matrix can be obtained; by orthogonalizing the 3×3 second prediction matrix, a corresponding 3×3 second orthogonal matrix can be obtained, which represents the prediction result of the digital cardiac plane representation information corresponding to the second training sample; by converting the 3×3 second orthogonal matrix into a corresponding 1×9 third prediction vector, the second label of the second training sample (also a 3×3 matrix) can be converted into a corresponding 1×9 first digital cardiac plane representation vector, which makes it easier to calculate the second loss function value. It should be noted that, as those skilled in the art will understand, the second threshold can be set according to actual circumstances, and the present invention does not limit this. When the value of the second loss function is greater than or equal to the second threshold, training needs to continue. It should also be noted that, as those skilled in the art will understand, the second orthogonal matrix is ​​an orthogonal matrix similar to the first orthogonal matrix mentioned above. The second orthogonal matrix also uses the trigonometric function values ​​of the Euler angles of the cross-section corresponding to the target echocardiogram training image in the space of the digital three-dimensional heart model to represent the prediction result of the corresponding digital heart plane representation information.

[0094] The formula for calculating the value of the second loss function is as follows:

[0095]

[0096] Where Loss2 represents the value of the second loss function, and n represents the number of the second training samples. Let j represent the j-th parameter in the third prediction vector corresponding to the i-th second training sample. It represents the j-th parameter in the first digital heart plane representation vector corresponding to the i-th second training sample.

[0097] Further, the orthogonalization process of the second prediction matrix to obtain the corresponding second orthogonal matrix includes:

[0098] Singular value decomposition is performed on the second prediction matrix, and the corresponding second orthogonal matrix is ​​obtained based on the singular value decomposition result of the second prediction matrix.

[0099] Specifically, for more information on how to obtain the corresponding second orthogonal matrix based on the singular value decomposition result of the second prediction matrix, please refer to the relevant content above on how to obtain the corresponding first orthogonal matrix based on the singular value decomposition result of the first prediction matrix, so it will not be repeated here.

[0100] In one exemplary embodiment, training the pre-built digital heart plane recognition model using the first training samples and gradient descent until a first preset training termination condition is met includes:

[0101] The echocardiogram training image of the first training sample is input into the pre-built digital cardiac plane recognition model to obtain the corresponding fourth prediction vector;

[0102] The fourth prediction vector is converted into the corresponding third prediction matrix, and the third prediction matrix is ​​orthogonalized to obtain the corresponding third orthogonal matrix.

[0103] The third orthogonal matrix is ​​transformed into the corresponding fifth prediction vector, and the first label of the first training sample is transformed into the corresponding second digital heart plane representation vector.

[0104] Calculate the first loss function value between the fifth prediction vector and the second digital heart plane representation vector;

[0105] Determine whether the value of the first loss function is less than the first threshold. If so, end the training.

[0106] Specifically, the fourth prediction vector is a 1×9 vector. By transforming the 1×9 fourth prediction vector, a corresponding 3×3 third prediction matrix can be obtained. By orthogonalizing the 3×3 third prediction matrix, a corresponding 3×3 third orthogonal matrix can be obtained. This 3×3 third orthogonal matrix represents the prediction result of the digital heart plane representation information corresponding to the first training sample. By converting the 3×3 third orthogonal matrix into a corresponding 1×9 fifth prediction vector, the first label of the first training sample (also a 3×3 matrix) is converted into a corresponding 1×9 second digital heart plane representation vector, which facilitates the calculation of the first loss function value. It should be noted that, as those skilled in the art will understand, the first threshold can be set according to actual conditions, and this invention does not limit it. When the first loss function value is greater than or equal to the first threshold, training needs to continue. It should also be noted that, as those skilled in the art will understand, the third orthogonal matrix is ​​an orthogonal matrix similar to the first orthogonal matrix mentioned above. The third orthogonal matrix also uses the trigonometric function value of the Euler angle of the digital cardiac plane in the space where the cross-section corresponding to the echocardiogram training image is located to represent the prediction result of the digital cardiac plane representation information.

[0107] The formula for calculating the first loss function value is as follows:

[0108]

[0109] Where Loss1 represents the value of the first loss function, and m represents the number of the first training samples. This represents the j-th parameter in the fifth prediction vector corresponding to the i-th first training sample. It represents the j-th parameter in the second digital heart plane representation vector corresponding to the i-th first training sample.

[0110] Further, the orthogonalization process of the third prediction matrix to obtain the corresponding third orthogonal matrix includes:

[0111] Singular value decomposition is performed on the third prediction matrix, and the corresponding third orthogonal matrix is ​​obtained based on the singular value decomposition result of the third prediction matrix.

[0112] Specifically, for more information on how to obtain the corresponding third orthogonal matrix based on the singular value decomposition result of the third prediction matrix, please refer to the relevant content above on how to obtain the corresponding first orthogonal matrix based on the singular value decomposition result of the first prediction matrix, so it will not be repeated here.

[0113] In one exemplary embodiment, the step of using a pre-trained digital cardiac plane recognition model to recognize the acquired current frame echocardiogram to obtain digital cardiac plane representation information of the cross-section corresponding to the current frame echocardiogram in the space of a pre-established digital three-dimensional heart model includes:

[0114] A pre-trained digital cardiac plane recognition model is used to identify the current frame of echocardiogram to obtain the corresponding fourth prediction matrix;

[0115] The fourth prediction matrix is ​​orthogonalized to obtain the corresponding fourth orthogonal matrix;

[0116] Based on the fourth orthogonal matrix, obtain the digital cardiac plane representation information of the section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model.

[0117] Specifically, by inputting the current frame echocardiogram into the digital cardiac plane recognition model, a 1×9 initial vector representing the digital cardiac plane corresponding to the current frame echocardiogram can be output. This initial vector can be transformed to obtain a corresponding 3×3 fourth prediction matrix. By orthogonalizing the 3×3 fourth prediction matrix, a corresponding 3×3 fourth orthogonal matrix can be obtained. This 3×3 fourth orthogonal matrix represents the digital cardiac plane representation information of the section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model. It should be noted that, as those skilled in the art will understand, the fourth orthogonal matrix is ​​an orthogonal matrix similar to the first orthogonal matrix mentioned above. The fourth orthogonal matrix also uses the trigonometric function values ​​of the Euler angles of the section corresponding to the current frame echocardiogram in the space of the digital three-dimensional heart model to represent the corresponding digital cardiac plane representation information.

[0118] In one exemplary embodiment, the step of obtaining the target motion information of the ultrasound probe to be guided, based on the digital cardiac plane representation information corresponding to the current frame echocardiogram and the pre-acquired digital cardiac plane representation information of the target standard section under the target acoustic window in the space where the digital three-dimensional heart model is located, includes:

[0119] Based on the digital cardiac plane representation information corresponding to the current frame echocardiogram and the digital cardiac plane representation information corresponding to the target standard section, calculate the transformation matrix between the digital cardiac plane corresponding to the current frame echocardiogram and the digital cardiac plane corresponding to the target standard section;

[0120] Based on the transformation matrix, the target motion information of the ultrasound probe to be guided is obtained.

[0121] Specifically, assuming the fourth orthogonal matrix is ​​Rc, that is, the digital cardiac plane representation information of the section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model, is represented by Rc. C This indicates that the target standard section represents the digital heart plane in the space where the digital three-dimensional heart model is located, expressed as matrix R. S Therefore, the transformation matrix R between the digital cardiac plane corresponding to the current frame echocardiogram and the digital cardiac plane corresponding to the target standard section can be calculated according to formula (5):

[0122] R = R S R C -1 (5)

[0123] Furthermore, after calculating the transformation matrix R, the target motion quantities of the ultrasound probe to be guided can be calculated according to formula (6), including the target tilt angle Δα', the target rotation angle Δβ', and the target swing angle Δγ':

[0124]

[0125] It should be noted that, as those skilled in the art will understand, after calculating the target tilt angle Δα', target rotation angle Δβ', and target swing angle Δγ', the operator, based on the current posture of the ultrasound probe to be guided, sequentially tilts the ultrasound probe to be guided by Δα', rotates it by Δβ', and swings it by Δγ', thereby moving the ultrasound probe to be guided to the direction of the target standard section. It should also be noted that, as those skilled in the art will understand, a 11 To a 33 The transformation matrix represents 9 parameters in R, a 11 To a 33 The specific value can be calculated using formula (5).

[0126] In one exemplary embodiment, the echocardiographic window scanning guided method further includes:

[0127] The digital cardiac plane corresponding to the current frame of echocardiogram, the digital cardiac plane corresponding to the target standard section, and the target motion information of the ultrasound probe to be guided are output and displayed.

[0128] Therefore, by displaying the digital cardiac plane corresponding to the current frame of echocardiography, the digital cardiac plane corresponding to the target standard section, and the target motion information of the ultrasound probe to be guided on the display, the operator can more intuitively operate the ultrasound probe to be guided to move to the direction corresponding to the target standard section based on the direction of the digital cardiac plane corresponding to the current frame of echocardiography, the direction of the digital cardiac plane corresponding to the target standard section, and the target motion information.

[0129] In one exemplary embodiment, the orthogonalization of the fourth prediction matrix to obtain a corresponding fourth orthogonal matrix includes:

[0130] Singular value decomposition is performed on the fourth prediction matrix, and the corresponding fourth orthogonal matrix is ​​obtained based on the singular value decomposition result of the fourth prediction matrix.

[0131] Specifically, for more information on how to obtain the corresponding fourth orthogonal matrix based on the singular value decomposition result of the fourth prediction matrix, please refer to the relevant content above on how to obtain the corresponding first orthogonal matrix based on the singular value decomposition result of the first prediction matrix, so it will not be repeated here.

[0132] In one exemplary embodiment, the method further includes:

[0133] Based on the digital heart plane representation information of all standard sections under the target acoustic window in the space of the digital three-dimensional heart model obtained in advance, the position information of the intersection points of all standard sections under the target acoustic window is obtained.

[0134] Based on the position information of the intersection points of all standard sections under the target acoustic window, the position information of the ultrasound probe to be guided in the space of the digital three-dimensional heart model is determined;

[0135] Based on the position information of the ultrasound probe to be guided in the space of the digital three-dimensional heart model, the digital cardiac plane corresponding to the current frame of echocardiogram and the digital cardiac plane corresponding to the target standard section are translated to the position of the ultrasound probe to be guided in the space of the digital three-dimensional heart model.

[0136] Specifically, based on the digital cardiac plane representation information of each standard section under the target acoustic window in the space of the digital three-dimensional heart model, the intersection point of all standard sections under the target acoustic window can be calculated. The location of this intersection point is the position of the ultrasound probe to be guided in the space of the digital three-dimensional heart model. Therefore, by translating the digital cardiac plane corresponding to the current frame of echocardiography and the digital cardiac plane corresponding to the target standard section, the center of the digital cardiac plane corresponding to the current frame of echocardiography and the center of the digital cardiac plane corresponding to the target standard section are translated to the position of the ultrasound probe to be guided in the space of the digital three-dimensional heart model. This allows the operator to clearly understand the direction of the digital cardiac plane corresponding to the current frame of echocardiography and the digital cardiac plane corresponding to the target standard section relative to the position of the ultrasound probe to be guided in the space of the digital three-dimensional heart model. This makes it easier for non-ultrasound professionals to easily move the ultrasound probe to the direction of the target standard section based on the target tilt angle Δα', target rotation angle Δβ', and target swing angle Δγ' calculated above.

[0137] It should be noted that, as those skilled in the art will understand, the intersection of the planes calculated above based on the planar representation of each standard section under the target acoustic window in the space where the digital three-dimensional heart model is located (i.e., based on the three Euler angles α, β, γ of the digital cardiac plane corresponding to each standard section under the target acoustic window in the space where the digital three-dimensional heart model is located) is the position of the ultrasound probe to be guided in the space where the digital three-dimensional heart model is located. It should also be noted that, as those skilled in the art will understand, the center of the digital cardiac plane corresponding to the current frame of echocardiography refers to the intersection of the digital cardiac plane corresponding to the previous frame of echocardiography and its base normal vector (i.e., the normal vector passing through the origin of the space where the digital three-dimensional heart model is located), and the center of the digital cardiac plane corresponding to the target standard section refers to the intersection of the digital cardiac plane corresponding to the target standard section and its base normal vector.

[0138] Based on the same inventive concept, this invention also provides an ultrasound imaging system, please refer to... Figure 5 The diagram illustrates a block structure of an ultrasound imaging system according to an embodiment of the present invention. Figure 5As shown, the ultrasound imaging system provided by this invention includes an ultrasound probe 100 and a controller 200 connected in communication. The ultrasound probe 100 is configured to acquire echocardiographic images and transmit them to the controller 200. The controller 200 is configured to implement the echocardiographic acoustic window scanning guidance method described above. Therefore, the ultrasound imaging system provided by this invention, by introducing a digital three-dimensional heart model, uses the digital cardiac plane representation information corresponding to the target standard section and the digital cardiac plane representation information corresponding to the current section (i.e., the section corresponding to the current frame of the echocardiographic image) as references to guide the movement of the ultrasound probe 100 during echocardiographic acoustic window (e.g., the apical acoustic window) scanning. This allows non-ultrasound professionals to easily perform echocardiographic acoustic window (e.g., the apical acoustic window) scanning, thereby reducing the workload of senior ultrasound physicians. Furthermore, this invention can guide the movement of the ultrasound probe 100 without introducing additional auxiliary devices or making any modifications to the ultrasound probe 100, thus further reducing the operational difficulty for ultrasound scanners and saving costs.

[0139] Please continue to refer to this. Figure 5 ,like Figure 5 As shown, in an exemplary embodiment, the ultrasound imaging system provided by the present invention further includes a display 300 communicatively connected to the controller 200. The display 300 is configured to display at least one of the echocardiogram, the digital three-dimensional heart model, the digital cardiac plane corresponding to the target standard section, and the digital cardiac plane corresponding to the current frame echocardiogram. Preferably, the display 300 is configured to simultaneously display the current frame echocardiogram, the digital three-dimensional heart model, the digital cardiac plane corresponding to the target standard section, and the digital cardiac plane corresponding to the current frame echocardiogram. This allows for better guidance for non-ultrasound professionals to move the ultrasound probe 100 in the direction corresponding to the target standard section.

[0140] Based on the same inventive concept, the present invention also provides an electronic device, please refer to... Figure 6 The diagram illustrates a block structure of an electronic device according to an embodiment of the present invention. Figure 6As shown, the electronic device includes a processor 101 and a memory 103. The memory 103 stores a computer program. When the computer program is executed by the processor 101, it implements the echocardiography acoustic window scanning guided method described above. Since the electronic device provided by this invention and the echocardiography acoustic window scanning guided method provided by this invention belong to the same inventive concept, the electronic device provided by this invention has all the advantages of the echocardiography acoustic window scanning guided method provided by this invention. For details, please refer to the relevant descriptions above. Therefore, the beneficial effects of the electronic device provided by this invention will not be elaborated upon here.

[0141] like Figure 6 As shown, the electronic device also includes a communication interface 102 and a communication bus 104, wherein the processor 101, the communication interface 102, and the memory 103 communicate with each other via the communication bus 104. The communication bus 104 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 104 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 102 is used for communication between the aforementioned electronic device and other devices.

[0142] The processor 101 referred to in this invention can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 101 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines.

[0143] The memory 103 can be used to store the computer program. The processor 101 implements various functions of the electronic device by running or executing the computer program stored in the memory 103 and calling the data stored in the memory 103.

[0144] The memory 103 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0145] This invention also provides a readable storage medium storing a computer program that, when executed by a processor, can implement the echocardiographic acoustic window scanning guided method described above. Since the readable storage medium provided by this invention and the echocardiographic acoustic window scanning guided method provided by this invention belong to the same inventive concept, the readable storage medium provided by this invention possesses all the advantages of the echocardiographic acoustic window scanning guided method provided by this invention. For details, please refer to the relevant descriptions above; therefore, the beneficial effects of the readable storage medium provided by this invention will not be elaborated upon here.

[0146] The readable storage medium provided in the embodiments of the present invention can be any combination of one or more computer-readable media. The readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer hard disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0147] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0148] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] In summary, the echocardiography acoustic window scanning guidance method, ultrasound imaging system, electronic device, and readable storage medium provided by the present invention have the following advantages: The present invention first acquires the current frame echocardiogram image captured by the ultrasound probe to be guided in the target acoustic window; then, it uses a pre-trained digital cardiac plane recognition model to recognize the acquired current frame echocardiogram image to obtain the digital cardiac plane representation information of the section corresponding to the current frame echocardiogram image in the space of the pre-established digital three-dimensional heart model; finally, based on the digital cardiac plane representation information corresponding to the current frame echocardiogram image and the pre-acquired digital cardiac plane representation information of the target standard section under the target acoustic window in the space of the digital three-dimensional heart model, it obtains the target motion information of the ultrasound probe to be guided, so as to guide the ultrasound probe to be guided to move to the direction corresponding to the target standard section. Therefore, this invention, by introducing a digital three-dimensional heart model and using the digital cardiac plane representation information corresponding to the target standard section and the digital cardiac plane representation information corresponding to the current section (i.e., the section corresponding to the current frame of echocardiography image) as references, provides guidance for the movement of the ultrasound probe during echocardiographic acoustic window (e.g., the apical acoustic window) scanning. This allows non-ultrasound professionals to easily perform echocardiographic acoustic window (e.g., the apical acoustic window) scanning, thereby reducing the workload of senior ultrasound physicians. Furthermore, this invention guides ultrasound probe movement without introducing additional auxiliary devices or modifying the ultrasound probe, further reducing the operational difficulty for ultrasound scanners and saving costs.

[0150] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0151] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the invention without departing from the spirit and scope of the invention. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for guiding echocardiographic window scanning, characterized in that, include: Acquire the current frame of echocardiogram image captured by the ultrasound probe to be guided within the target acoustic window; A pre-trained digital cardiac plane recognition model is used to identify the acquired current frame echocardiogram to obtain the digital cardiac plane representation information of the section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model. The digital cardiac plane representation information corresponding to the section is represented by trigonometric functions related to the three Euler angles of the digital cardiac plane corresponding to the section. The three Euler angles correspond to the tilt angle, rotation angle and swing angle of the ultrasound probe, respectively. Based on the digital cardiac plane representation information corresponding to the current frame echocardiogram and the digital cardiac plane representation information of the target standard section under the target acoustic window in the space of the digital three-dimensional heart model, the target motion amount information of the ultrasound probe to be guided is obtained to guide the ultrasound probe to be guided to move to the direction corresponding to the target standard section. The target motion amount information includes the target tilt angle, the target rotation angle and the target swing angle. The method further includes: Based on the digital heart plane representation information of all standard sections under the target acoustic window in the space of the digital three-dimensional heart model obtained in advance, the position information of the intersection points of all standard sections under the target acoustic window is obtained. Based on the position information of the intersection points of all standard sections under the target acoustic window, the position information of the ultrasound probe to be guided in the space of the digital three-dimensional heart model is determined; Based on the position information of the ultrasound probe to be guided in the space of the digital three-dimensional heart model, the digital cardiac plane corresponding to the current frame of echocardiogram and the digital cardiac plane corresponding to the target standard section are translated to the position of the ultrasound probe to be guided in the space of the digital three-dimensional heart model.

2. The echocardiography window scanning guided method according to claim 1, characterized in that, The digital heart-plane recognition model is trained through the following steps: Multiple first training samples are obtained, wherein each first training sample includes an echocardiogram training image and its corresponding first label. The first label represents the digital cardiac plane representation information of the cross section corresponding to the echocardiogram training image in the space where the digital three-dimensional heart model is located. All the echocardiogram training images are acquired under the target acoustic window, and for each standard cross section under the target acoustic window, at least one cross section corresponding to the echocardiogram training image is the standard cross section. The pre-built digital heart plane recognition model is trained based on the first training sample until a first preset training termination condition is met. The first preset training termination condition is that the number of training iterations reaches a preset first iteration number or the corresponding first loss function value is less than a first threshold.

3. The echocardiography window scanning guided method according to claim 2, characterized in that, The acquisition of multiple first training samples includes: An ultrasound probe with a fixed inertial measurement unit is used to continuously scan the target acoustic window to obtain echocardiographic training images of various standard and non-standard sections under the target acoustic window, as well as the attitude information of the ultrasound probe when acquiring each of the echocardiographic training images, wherein the attitude information is acquired by the inertial measurement unit. For each of the aforementioned echocardiographic training images, based on the posture information corresponding to the echocardiographic training image and the pre-acquired digital cardiac plane regression model, the digital cardiac plane representation information of the cross section corresponding to the echocardiographic training image in the space where the digital three-dimensional heart model is located is obtained, and the echocardiographic training image and its corresponding digital cardiac plane representation information are combined to form an example of the first training sample.

4. The echocardiography window scanning guided method according to claim 3, characterized in that, The digital cardiac plane regression model is a pre-trained neural network regression model. The step of obtaining the digital cardiac plane representation information of the cross-section corresponding to the echocardiogram training image in the space of the digital three-dimensional heart model, based on the pose information corresponding to the echocardiogram training image and the pre-acquired digital cardiac plane regression model, includes: The corresponding rotational affine matrix is ​​obtained based on the posture information corresponding to the echocardiogram training image, and the rotational affine matrix is ​​converted into the corresponding rotational affine vector. The affine rotation vector is input into the digital heart-plane regression model to obtain the corresponding prediction vector; The prediction vector is converted into a prediction matrix corresponding to the echocardiogram training image, and the prediction matrix corresponding to the echocardiogram training image is orthogonalized to obtain the orthogonal matrix corresponding to the echocardiogram training image, thereby obtaining the digital cardiac plane representation information of the cross-section corresponding to the echocardiogram training image in the space where the digital three-dimensional heart model is located.

5. The echocardiography window scanning guided method according to claim 4, characterized in that, The digital cardiac plane regression model is trained through the following steps: The echocardiogram training image with the cross-section of the standard cross-section under the target acoustic window is selected as the target echocardiogram training image; For each target echocardiogram training image, the digital cardiac plane representation information of the standard section corresponding to the target echocardiogram training image in the space where the digital three-dimensional heart model is located is used as a second label and the pose information corresponding to the target echocardiogram training image to form a second training sample. The pre-built neural network regression model is trained based on the second training sample until the second preset training termination condition is met, so as to obtain the digital heart plane regression model. The second preset training termination condition is that the number of training iterations reaches the preset second iteration number or the corresponding second loss function value is less than the second threshold.

6. The echocardiography window scanning guided method according to claim 4, characterized in that, The orthogonalization process performed on the prediction matrix corresponding to the echocardiographic training image to obtain the orthogonal matrix corresponding to the echocardiographic training image includes: Singular value decomposition is performed on the prediction matrix corresponding to the echocardiogram training image, and the orthogonal matrix corresponding to the echocardiogram training image is obtained based on the singular value decomposition result of the prediction matrix corresponding to the echocardiogram training image.

7. The echocardiography window scanning guided method according to claim 1, characterized in that, The process of using a pre-trained digital cardiac plane recognition model to identify the acquired current frame echocardiogram image to obtain digital cardiac plane representation information of the cross-section corresponding to the current frame echocardiogram image in the space of the pre-established digital three-dimensional heart model includes: A pre-trained digital cardiac plane recognition model is used to identify the current frame of echocardiogram to obtain the prediction matrix corresponding to the current frame of echocardiogram. The prediction matrix corresponding to the current frame echocardiogram is orthogonalized to obtain the digital cardiac plane representation information of the cross-section corresponding to the current frame echocardiogram in the space of the pre-established digital three-dimensional heart model.

8. An ultrasound imaging system, characterized in that, The method includes an ultrasound probe and a controller with a communication connection, the ultrasound probe being configured to acquire echocardiogram images and transmit them to the controller, the controller being configured to implement the echocardiogram acoustic window scanning guided method according to any one of claims 1 to 7.

9. The ultrasound imaging system according to claim 8, characterized in that, It also includes a display communicatively connected to the controller, the display being configured to display at least one of the echocardiogram, the digital three-dimensional heart model, the digital cardiac plane corresponding to the target standard section, and the digital cardiac plane corresponding to the current frame echocardiogram.

10. An electronic device, characterized in that, The method includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the echocardiography window scanning guided method according to any one of claims 1 to 7.

11. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements the echocardiography window scanning guided method according to any one of claims 1 to 7.

Images