Method of annotating ultrasound images and ultrasound imaging system

Abstract

An annotation method of an ultrasound image and an ultrasound imaging system, the method comprising: acquiring and displaying an ultrasound image of a four-chamber heart section of a target object; determining a position corresponding to a cross structure of the four-chamber heart section in the ultrasound image based on the ultrasound image or ultrasound data corresponding to the ultrasound image; and generating a plurality of annotation marks representing a plurality of anatomical structures in the four-chamber heart section on the ultrasound image with the position corresponding to the cross structure as a reference, wherein centers of the plurality of annotation marks correspond to the reference, and distances and / or angles of the plurality of annotation marks relative to the centers of the plurality of annotation marks change synchronously with an adjusting operation. The present application can annotate a plurality of anatomical structures in the ultrasound image of the four-chamber heart section at one time, and improve the annotation efficiency of the ultrasound image.

Description

Technical Field

[0001] This invention relates to the field of ultrasound imaging technology, and more specifically to a method for annotating ultrasound images and an ultrasound imaging system. Background Technology

[0002] Ultrasound, as a safe, inexpensive, and highly reliable modern medical imaging technology, has been widely used in various departments such as gynecology, obstetrics, and cardiology, providing a reliable tool for the diagnosis of human diseases. With the continuous development and advancement of technology, ultrasound-assisted diagnostic technology is increasingly being applied in actual clinical diagnosis, not only significantly improving the efficiency of clinical work for doctors but also further expanding the scope of application of ultrasound technology.

[0003] Prenatal screening, as one of the most important examinations for ruling out fetal malformations, plays a crucial role in ensuring normal fetal development. The four-chamber view, as one of the most important standard cardiac views in prenatal screening, is the primary basis for identifying congenital heart diseases such as atrial septal defect, ventricular septal defect, single ventricle, and mitral or tricuspid valve dysplasia, and plays a vital role in assessing whether the fetal heart is developing normally. The four-chamber view generally includes multiple anatomical structures such as the left atrium, left ventricle, right atrium, and right ventricle. Traditionally, annotation is done manually by physicians, but due to the numerous anatomical structures in this view, annotation is time-consuming and labor-intensive, reducing the actual work efficiency of clinicians. Summary of the Invention

[0004] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key features and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0005] One embodiment of the present invention provides a method for annotating ultrasound images, the method comprising:

[0006] Acquire and display ultrasound images of a four-chamber view of the target object;

[0007] Based on the ultrasound image or the ultrasound data corresponding to the ultrasound image, determine the position of the cross-shaped structure of the four-chamber view in the ultrasound image;

[0008] Using the position corresponding to the cross structure as a reference, multiple annotation marks representing multiple anatomical structures in the four-chamber view are generated on the ultrasound image. The center of the multiple annotation marks corresponds to the reference, and the distance and / or angle of the multiple annotation marks relative to the center of the multiple annotation marks can change synchronously with the adjustment operation.

[0009] In some embodiments, determining the position of the cruciform structure of the four-chamber view in the ultrasound image based on the ultrasound image of the four-chamber view or the ultrasound data corresponding to the ultrasound image includes:

[0010] Based on the ultrasound images of the four-chamber view or the ultrasound data corresponding to the ultrasound images, determine the regions corresponding to the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound images;

[0011] The position of the cross-shaped structure in the ultrasound image is determined by the intersection point of the corresponding regions of the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image.

[0012] In some embodiments, determining the position of the cross-shaped structure in the ultrasound image based on the intersection point of the regions corresponding to the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image includes:

[0013] In the ultrasound image, determine the center points of the regions corresponding to the left atrium, left ventricle, right atrium, and right ventricle, respectively;

[0014] Determine the line connecting the center point of the region corresponding to the left atrium and the center point of the region corresponding to the right ventricle, and the line connecting the center point of the region corresponding to the left ventricle and the center point of the region corresponding to the right atrium;

[0015] The intersection of the two lines is determined as the position of the cross structure in the ultrasound image.

[0016] In some embodiments, determining the position of the cruciform structure of the four-chamber view in the ultrasound image based on the ultrasound image of the four-chamber view or the ultrasound data corresponding to the ultrasound image includes:

[0017] The ultrasound images of the four-chamber view or the ultrasound data corresponding to the ultrasound images are identified based on a machine learning model to determine the region corresponding to the cruciate structure.

[0018] The position of the cross-shaped structure is determined based on the center point of the region corresponding to the cross-shaped structure.

[0019] In some embodiments, determining the position of the cruciform structure of the four-chamber view in the ultrasound image based on the ultrasound image of the four-chamber view or the ultrasound data corresponding to the ultrasound image includes:

[0020] The location operation is performed based on the ultrasound image reception to locate the position corresponding to the cross structure, and the position corresponding to the cross structure is determined according to the location operation.

[0021] In some embodiments, the plurality of annotation markers include a left atrial annotation marker, a left ventricular annotation marker, a right atrial annotation marker, and a right ventricular annotation marker, wherein the left atrial annotation marker, the left ventricular annotation marker, the right atrial annotation marker, and the right ventricular annotation marker are text markers or graphic markers.

[0022] In some embodiments, the distances between the left atrial annotation mark, the left ventricular annotation mark, the right atrial annotation mark, and the right ventricular annotation mark and the reference are equal, and the line connecting the left atrial annotation mark and the right ventricular annotation mark is perpendicular to the line connecting the left ventricular annotation mark and the right atrial annotation mark.

[0023] In some embodiments, the adjustment operation includes at least one of the following:

[0024] The positions of the centers of the plurality of annotation marks on the ultrasound image are fixed, and the distance between the plurality of annotation marks and their centers is adjusted synchronously.

[0025] The centers of the plurality of annotation marks are fixed in position on the ultrasound image, and the angles of the plurality of annotation marks relative to their centers are adjusted synchronously.

[0026] In some embodiments, the method further includes:

[0027] Two opposing annotation markers among the plurality of annotation markers may be flipped relative to the center of the plurality of annotation markers during a flip adjustment operation; and / or,

[0028] The multiple annotation markers can move synchronously as the center position of the multiple annotation markers is adjusted.

[0029] In some embodiments, the method further includes:

[0030] Display operation controls for adjusting the plurality of annotation marks, and receive adjustment operations through the operation controls to adjust the distance and angle of the plurality of annotation marks relative to the reference.

[0031] Another aspect of this invention provides a method for annotating ultrasound images, the method comprising:

[0032] Acquire and display ultrasound images of a four-chamber view of the target object;

[0033] In response to the annotation operation, multiple annotation markers characterizing multiple anatomical structures in the four-chamber view are generated, wherein the distance and / or angle of the multiple annotation markers relative to the center of the multiple annotation markers can change synchronously with the adjustment operation.

[0034] In some embodiments, the plurality of annotation markers include a left atrial annotation marker, a left ventricular annotation marker, a right atrial annotation marker, and a right ventricular annotation marker, wherein the left atrial annotation marker, the left ventricular annotation marker, the right atrial annotation marker, and the right ventricular annotation marker are text markers or graphic markers.

[0035] In some embodiments, the method further includes:

[0036] The position of the cross-shaped structure of the four-chamber view in the ultrasound image is determined, or the position of the intersection of the cross-shaped structure of the four-chamber view in the ultrasound image is determined, and the position is indicated by a first identifier displayed on the ultrasound image.

[0037] In some embodiments, the adjustment operation includes at least one of the following:

[0038] The center positions of the multiple annotation marks are fixed, and the distance between the multiple annotation marks and their centers is adjusted synchronously.

[0039] The center positions of the multiple annotation markers are fixed, and the angles of the multiple annotation markers relative to their centers are adjusted synchronously.

[0040] In some embodiments, the method further includes:

[0041] Two opposing annotation markers among the plurality of annotation markers may be flipped relative to the center of the plurality of annotation markers during a flip adjustment operation; and / or,

[0042] The multiple annotation markers can move synchronously as the center position of the multiple annotation markers is adjusted.

[0043] A third aspect of the present invention provides an ultrasound imaging system, the ultrasound imaging system comprising:

[0044] Ultrasonic probe;

[0045] A transmitting circuit is used to excite the ultrasound probe to emit ultrasound waves toward the heart of the target object;

[0046] A receiving circuit is used to control the ultrasonic probe to receive the echo of the ultrasonic wave and obtain the echo signal;

[0047] A processor is configured to generate an ultrasound image of a four-chamber view of a target object based on the echo signal, and to perform the ultrasound image annotation method as described above.

[0048] A display for showing the ultrasound images.

[0049] The ultrasound image annotation method and ultrasound imaging system of this invention use the position corresponding to the cross structure as a reference, and can annotate multiple anatomical structures in the ultrasound image of the four-chamber view at one time, thereby improving the annotation efficiency of ultrasound images. Attached Figure Description

[0050] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0051] Figure 1 A structural block diagram of an ultrasound imaging system according to an embodiment of the present invention is shown;

[0052] Figure 2 A schematic flowchart illustrating a method for annotating ultrasound images according to an embodiment of the present invention is shown.

[0053] Figure 3A and Figure 3B A schematic diagram showing the position of a cross-shaped structure according to an embodiment of the present invention;

[0054] Figure 4A , Figure 4B and Figure 4C A schematic diagram showing annotation marks displayed in an ultrasound image according to an embodiment of the present invention is shown;

[0055] Figure 5A , Figure 5B , Figure 5C and Figure 5D A schematic diagram illustrating the adjustment operation of annotation markers according to an embodiment of the present invention is shown;

[0056] Figure 6 A schematic flowchart illustrating a method for annotating ultrasound images according to another embodiment of the present invention is shown. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0058] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0059] It should be understood that the invention can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, providing these embodiments will make the disclosure thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0060] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising” and / or “including,” when used in this specification, identify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.

[0061] To fully understand this invention, a detailed structure will be presented in the following description to illustrate the technical solution proposed by this invention. Optional embodiments of the invention are described in detail below; however, in addition to these detailed descriptions, the invention may have other embodiments.

[0062] Below, first refer to Figure 1 An ultrasound imaging system according to an embodiment of the present invention is described. Figure 1 A schematic structural block diagram of an ultrasound imaging system 100 according to an embodiment of the present invention is shown.

[0063] like Figure 1 As shown, the ultrasound imaging system 100 includes an ultrasound probe 110, a transmitting circuit 112, a receiving circuit 114, a processor 116, and a display 118. Further, the ultrasound imaging system may also include a transmit / receive selection switch 120 and a beamforming module 122. The transmitting circuit 112 and the receiving circuit 114 can be connected to the ultrasound probe 110 via the transmit / receive selection switch 120.

[0064] The ultrasonic probe 110 includes multiple transducer elements. These elements can be arranged in a row to form a linear array, or in a two-dimensional matrix to form a planar array. They can also form a convex array. Each transducer element is used to emit ultrasonic waves based on an excitation electrical signal, or to convert received ultrasonic waves into electrical signals. Therefore, each transducer element can be used to achieve the mutual conversion between electrical pulse signals and ultrasonic waves, thereby enabling the emission of ultrasonic waves to the target area of ​​the object being tested, and also to receive ultrasonic wave echoes reflected back from the tissue. During ultrasonic testing, the transmission and reception sequences can be used to control which transducer elements are used to emit ultrasonic waves and which are used to receive ultrasonic waves, or to control the transducer elements to be used in time-slotted manner for emitting ultrasonic waves or receiving ultrasonic wave echoes. Transducer elements participating in ultrasonic wave emission can be simultaneously excited by electrical signals, thus emitting ultrasonic waves simultaneously; alternatively, transducer elements participating in ultrasonic beam emission can be excited by several electrical signals with a certain time interval, thus continuously emitting ultrasonic waves with a certain time interval.

[0065] During ultrasound imaging, processor 116 controls transmitting circuit 112 to send a delayed-focused transmission pulse to ultrasound probe 110 via transmit / receive selection switch 120. Excited by the transmission pulse, ultrasound probe 110 emits an ultrasonic beam towards the tissue of the target area of ​​the object being measured. After a certain delay, it receives the ultrasonic echo reflecting back from the tissue of the target area, carrying tissue information, and converts this ultrasonic echo back into an electrical signal. Receiving circuit 114 receives the electrical signal generated by ultrasound probe 110, obtains the ultrasonic echo signal, and sends these ultrasonic echo signals to beamforming module 122. Beamforming module 122 performs focusing delay, weighting, and channel summation on the ultrasonic echo data before sending it to processor 116. Processor 116 performs signal detection, signal enhancement, data conversion, and logarithmic compression on the ultrasonic echo signal to form an ultrasound image. The ultrasound image obtained by processor 116 can be displayed on display 118 or stored in memory 124.

[0066] Optionally, the processor 116 can be implemented as software, hardware, firmware, or any combination thereof, and can use one or more application-specific integrated circuits (ASICs), one or more general-purpose integrated circuits, one or more microprocessors, one or more programmable logic devices, or any combination of the foregoing circuits and / or devices, or other suitable circuits or devices. Furthermore, the processor 116 can control other components in the ultrasound imaging system 100 to perform the corresponding steps of the methods in the various embodiments of this specification.

[0067] The display 118 is connected to the processor 116. The display 118 can be a touch screen, an LCD screen, or a separate display such as an LCD screen or a television, independent of the ultrasound imaging system 100. Alternatively, the display 118 can be the screen of an electronic device such as a smartphone or tablet, etc. The number of displays 118 can be one or more.

[0068] The display 118 can display the ultrasound images obtained by the processor 116. Furthermore, while displaying the ultrasound images, the display 118 can also provide a graphical user interface for human-machine interaction. One or more controlled objects can be set on the graphical interface, allowing the user to input operation commands using a human-machine interaction device to control these controlled objects and perform corresponding control operations. For example, icons can be displayed on the graphical interface, and the human-machine interaction device can be used to operate these icons to perform specific functions, such as drawing a region of interest bounding box on the ultrasound image.

[0069] Optionally, the ultrasound imaging system 100 may also include other human-machine interface devices besides the display 118, which are connected to the processor 116. For example, the processor 116 may be connected to the human-machine interface device via an external input / output port, which may be a wireless communication module, a wired communication module, or a combination of both. The external input / output port may also be based on USB, bus protocols such as CAN, and / or wired network protocols.

[0070] The human-computer interaction device may include an input device for detecting user input information. This input information may be, for example, control commands for the timing of ultrasound transmission / reception, operational input commands for drawing points, lines, or boxes on an ultrasound image, or other types of commands. The input device may include one or a combination of several of the following: a keyboard, mouse, scroll wheel, trackball, mobile input device (e.g., a mobile device with a touchscreen, a mobile phone, etc.), a multi-function knob, etc. The human-computer interaction device may also include an output device such as a printer.

[0071] The ultrasound imaging system 100 may also include a memory 124 for storing instructions executed by the processor, receiving ultrasound echoes, storing ultrasound images, etc. The memory may be a flash memory card, solid-state memory, hard disk, etc. It may be volatile and / or non-volatile memory, removable memory and / or non-removable memory, etc.

[0072] It should be understood that Figure 1 The components included in the ultrasound imaging system 100 shown are merely illustrative and may include more or fewer components. This invention is not limited thereto.

[0073] Below, we will refer to Figure 2 A method for annotating ultrasound images according to an embodiment of the present invention is described, which can be implemented in the ultrasound imaging system 100 described above. Figure 2 This is a schematic flowchart of an ultrasound image annotation method 200 according to an embodiment of the present invention.

[0074] like Figure 2 As shown, an ultrasound image annotation method 200 according to an embodiment of the present invention includes the following steps:

[0075] In step S210, an ultrasound image of the four-chamber view of the target object is acquired and displayed;

[0076] In step S220, based on the ultrasound image or the ultrasound data corresponding to the ultrasound image, the position of the cross-shaped structure of the four-chamber view in the ultrasound image is determined.

[0077] In step S230, using the position corresponding to the cross structure as a reference, multiple annotation marks representing multiple anatomical structures in the four-chamber view are generated on the ultrasound image, wherein the center of the multiple annotation marks corresponds to the reference, and the distance and / or angle of the multiple annotation marks relative to the center of the multiple annotation marks can change synchronously with the adjustment operation.

[0078] The ultrasound image annotation method 200 of this invention uses the position corresponding to the cross structure as a reference, and can annotate multiple anatomical structures in the ultrasound image of the four-chamber view at one time, thereby improving the annotation efficiency of ultrasound images.

[0079] Specifically, in step S210, an ultrasound image of the four-chamber view of the target object can be generated and displayed in real time, or a pre-generated ultrasound image stored in a storage medium can be extracted. The target object includes, but is not limited to, a fetus. The four-chamber view is a section that allows visualization of the four chamber structures of the heart (left atrium, left ventricle, right atrium, and right ventricle), specifically a plane that runs from the apex of the left chest wall to the base of the heart, transversely across the left and right atria and ventricles. Ultrasound images of the four-chamber view allow observation of the size and shape of each atrioventricular cavity, and examination of any defects in the atrial septum and ventricular septum, thus possessing significant clinical value.

[0080] Reference Figure 1During ultrasound imaging of the fetal heart, the transmitting circuit 112 sends a set of delayed-focused transmission pulses to the ultrasound probe 110 to excite the ultrasound probe 110 to emit ultrasound waves toward the fetal heart in the pregnant woman's abdomen. After a certain delay, the receiving circuit 114 controls the ultrasound probe 110 to receive the ultrasound echo reflected from the fetal heart, converting it into an electrical signal. The beamforming module 112 performs corresponding delay and weighted summation processing on the echo signals obtained from multiple transmissions and receptions to achieve beamforming. The signal is then sent to the processor 116 for processing such as logarithmic compression, dynamic range adjustment, and digital scan transformation to generate an ultrasound image of the four chambers of the fetal heart, which is then displayed on the display interface. For example, the display interface can also display operation controls for adding annotations to anatomical structures in the ultrasound image, and subsequent steps are triggered in response to operation commands on these controls.

[0081] After acquiring the ultrasound image of the four-chamber view, in step S220, based on the ultrasound image or the corresponding ultrasound data, the position of the cruciate structure in the four-chamber view is determined in the ultrasound image. The ultrasound image of the four-chamber view shows four chambers: the left and right atria and the left and right ventricles. The atrial septum, interventricular septum, mitral valve, and tricuspid valve form a cruciate structure. This structure is unique to the four-chamber view and is located at the center of multiple anatomical structures within the four-chamber view, allowing for accurate localization of each anatomical structure. Specifically, the position of the cruciate structure in the ultrasound image refers to the position of the intersection of the cruciate structure within the ultrasound image.

[0082] There are two methods for determining the location of the cruciate structure: manual and automatic. The manual method involves the user selecting the location of the cruciate structure in the ultrasound image. The ultrasound imaging system then uses this selection to determine the location of the cruciate structure based on the received ultrasound image data. For example, the user can be prompted to click a point on the ultrasound image using a keyboard, mouse, or other tools, drawing on clinical experience; this point is the intersection of the cruciate structure. Alternatively, the user can be prompted to select the boundaries of the cruciate structure; the area selected by the user is the region where the cruciate structure is located, and the center point of this region is the intersection of the cruciate structure.

[0083] Automatic acquisition methods utilize image recognition algorithms to automatically locate cross-shaped structures. These methods can be categorized into two types: one is based on traditional image recognition methods to detect the position of the cross-shaped structure; the other is to use machine learning methods to automatically locate the cross-shaped structure.

[0084] The principle of traditional image recognition methods is as follows: since the cross-shaped structure in the four-chamber view is located in the middle of the left and right atria and ventricles, and the myocardium in this area has a vertical feature, the position of the cross-shaped structure can be located based on this feature. Specifically, based on the ultrasound image of the four-chamber view or the ultrasound data corresponding to the ultrasound image, the corresponding regions of the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image are determined; the position of the cross-shaped structure in the ultrasound image is determined based on the intersection point of the corresponding regions of the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image.

[0085] For example, the left and right atria and ventricles can be identified first by segmentation or edge extraction methods. Segmentation methods include Otsu thresholding algorithm (OSTU), level set method, graph cut, etc., while edge extraction algorithms include Sobel operator, Roberts operator, Prewitt operator, etc. Then, using the feature information of the segmented regions or edges, the center points of the corresponding regions of the left atrium, left ventricle, right atrium, and right ventricle are determined respectively. The lines connecting the center points of the regions corresponding to the left atrium and the right ventricle are obtained, as well as the lines connecting the center points of the regions corresponding to the left ventricle and the right atrium. The intersection of the two lines is determined as the position of the cross structure in the ultrasound image.

[0086] The principle of machine learning-based methods is to construct a training sample library and use strategies such as target detection, point regression, or Gaussian kernel to determine the region corresponding to the cruciate structure in the ultrasound image of the four-chamber view. The center point of this region is the location of the cruciate structure.

[0087] Specifically, the method for constructing the sample library includes: for a four-chamber view ultrasound image, the bounding box coordinates can be used to estimate the range of the cross structure. Thus, a training sample library with a one-to-one correspondence between the four-chamber view ultrasound image and the bounding box coordinates can be constructed. The four-chamber view ultrasound image is the input, and the bounding box coordinates are the ground truth output.

[0088] After constructing the training sample library, a machine learning model is designed to regress the bounding box coordinates of the cross-shaped structure. The main architecture of the machine learning model includes convolutional layers, pooling layers, activation layers, fully connected layers, and a defined loss function. These layers are combined and stacked to learn features from the training samples. For any ultrasound image in the training sample library, it is input into the designed network model, which outputs an estimated bounding box coordinate. The model parameters are then optimized using the estimated value, the ground truth, and the designed loss function to train the machine learning model. Commonly used network models include MobileNet, ResNet, and RetinaNet, but the machine learning models that can be used are not limited to the above network model structures.

[0089] During the inference phase, the ultrasound image or ultrasound data of the four-chamber view obtained in step S210 is input into the trained machine learning model, and the coordinates of the bounding box of the cross structure can be obtained in real time. Based on the coordinates of the bounding box, the coordinates of the center point of the bounding box can be further obtained, thereby realizing the detection of the position of the cross structure in the ultrasound image.

[0090] It should be noted that the above method for locating the cruciate structure is only an example. Other image recognition methods can also be used to locate the cruciate structure in ultrasound images.

[0091] See Figure 3A and Figure 3B In some embodiments, after determining the position of the cross-shaped structure in the ultrasound image, a first identifier 301 can be displayed on the ultrasound image to indicate the position of the cross-shaped structure or its intersection in the ultrasound image. The first identifier 301 can be a circular identifier, a cross-shaped identifier, etc. Displaying the position of the cross-shaped structure through the first identifier 301 can help the user determine whether the determined position of the cross-shaped structure is accurate.

[0092] Next, in step S230, using the position corresponding to the cruciate structure as a reference, multiple annotation markers representing multiple anatomical structures in the four-chamber view are generated on the ultrasound image. The centers of these annotation markers correspond to the reference, and the distances and / or angles of the annotation markers relative to their centers can change synchronously with adjustment operations. This embodiment of the invention automatically generates multiple annotation markers based on the position of the cruciate structure, eliminating the need for users to manually input the names of each anatomical structure, thus improving annotation efficiency. Furthermore, the positions of the annotation markers, centered on the cruciate structure, conform to the actual anatomical structure, allowing users to adjust all annotation markers at once without individually adjusting each one. Moreover, adjusting a subset of annotation markers to the correct position ensures the accuracy of all marker positions, enabling users lacking clinical knowledge to annotate all anatomical structures.

[0093] For example, the centers of multiple annotation markers can be used as fixed anchor points, coinciding with the intersection points of the cross structure serving as a reference. Since the positions of the multiple annotation markers are accurately positioned based on the cross structure, on the one hand, the user's adjustment operations for positioning the multiple annotation markers can be reduced; on the other hand, when the adjustment operations are overall linkage adjustments such as scaling, rotating, and flipping of the multiple annotation markers, the synchronous change of the multiple annotation markers relative to their centers can ensure a higher degree of overall correspondence between the multiple annotation markers and each anatomical structure.

[0094] For example, the anatomical structures in a four-chamber view of the heart include the left atrium, left ventricle, right atrium, and right ventricle. Accordingly, multiple annotation markers include left atrial annotation markers, left ventricular annotation markers, right atrial annotation markers, and right ventricular annotation markers. The left atrial, left ventricular, right atrial, and right ventricular annotation markers are either textual or graphic markers. See, for example, [link to relevant documentation]. Figure 4A , Figure 4B and Figure 4C The annotation labels for the left atrium, left ventricle, right atrium, and right ventricle are LA, LV, RA, and RV, respectively. The annotation labels can also be "left atrium," "left ventricle," or other text or graphics that can represent the corresponding anatomical structures.

[0095] In addition to the above, the anatomical structures in the four-chamber view of the heart include the mitral valve, tricuspid valve, atrial septum, interventricular septum, right superior pulmonary vein, and descending aorta. Annotations may also include labels for these anatomical structures. All anatomical results can be annotated with text or graphics.

[0096] After identifying the location of the cruciform structure, multiple annotation markers are generated based on the intersection position of the cruciform structure. In this embodiment of the invention, the annotation markers of multiple anatomical structures can be regarded as a whole. The annotation markers that have not been manually adjusted by the user can be called the initial annotation markers. Because the ultrasound images of the four-chamber view are affected by factors such as fetal size, fetal position, ultrasound emission direction, and whether the doctor zooms the image during the acquisition process, the position of the initial annotation markers often does not correspond to the actual position of the anatomical structure. Therefore, the user needs to adjust the initial annotation markers to match the actual position of each annotation marker with the actual position of the anatomical structure.

[0097] In some embodiments, such as Figure 5A As shown, the left atrial, left ventricular, right atrial, and right ventricular annotation marks are equidistant from the reference point, and the line connecting the left and right atrial annotation marks is perpendicular to the line connecting the left and right ventricular annotation marks. In other words, the left atrial, left ventricular, right atrial, and right ventricular annotation marks are rotationally symmetrical about the reference point. During user adjustments to the annotation marks, the left atrial, left ventricular, right atrial, and right ventricular annotation marks remain rotationally symmetrical about the reference point, and their relative positions remain fixed.

[0098] like Figure 5B As shown, the adjustment operation for the annotation marks includes a scaling operation. During the scaling operation, the center of multiple annotation marks is fixed on the ultrasound image and the angle with the reference remains fixed. The user can simultaneously adjust the distance of multiple annotation marks relative to the reference. Figure 5B It shows the Figure 5A The annotation marks obtained by magnifying the initial annotation marks, in Figure 5B In the image, the left atrial, left ventricular, right atrial, and right ventricular annotation markers remain rotationally symmetrical around the reference, and the distance between them and the reference is magnified by the same factor. Through scaling, the annotation markers can be adapted to different fetal sizes and different ultrasound image sizes.

[0099] like Figure 5C As shown, the adjustment operation for annotation markers also includes a rotation operation. During the rotation operation, the centers of multiple annotation markers are fixed in position on the ultrasound image and their distance from the reference remains fixed. The user can simultaneously adjust the angles of multiple annotation markers relative to the reference. Figure 5C It shows the Figure 5A The annotation marks obtained after rotating the initial annotation marks, in Figure 5CIn the image, the left atrial, left ventricular, right atrial, and right ventricular annotation markers remain rotationally symmetrical around the reference point and have been rotated by the same angle relative to the reference point. Scaling operations allow the annotation markers to be adapted to different fetal positions and imaging angles.

[0100] like Figure 5D As shown, the adjustment operation for annotation markers also includes a flip operation. During the rotation operation, the center positions of multiple annotation markers on the ultrasound image are fixed, and their distances from and angles relative to the reference are also fixed. Users can mirror the positions of the annotation markers, so that two opposite annotation markers can be flipped relative to the reference during the flip adjustment operation. Figure 5D It shows the Figure 5A The comment tags obtained by flipping the initial comment tags, in Figure 5A In the middle, the right atrial annotation marker is located to the left of the baseline, and the left ventricular annotation marker is located to the right of the baseline; while in... Figure 5D In the image, the left ventricular annotation marker is located to the left of the reference, and the right atrial annotation marker is located to the right of the reference. Similarly, the user can also flip the right ventricular and left atrial annotation markers. This flipping operation allows the annotation markers to adapt to different fetal positions and imaging orientations. Two opposing annotation markers can be flipped relative to the center of the plurality of annotation markers during the flipping adjustment operation.

[0101] In some embodiments, the adjustment operation of annotation markers may further include a movement operation. During the movement operation, multiple annotation markers may move synchronously following the position adjustment operation of the centers of multiple annotation markers. For example, when the identification or labeling of the position of the cross structure is inaccurate, the positions of multiple annotation markers will shift accordingly. The user can move multiple annotation markers synchronously, and the relative distance and angle between the multiple annotation markers remain unchanged during the movement.

[0102] For example, a user interaction method for receiving adjustment operations may include displaying operation controls for adjusting multiple annotation markers and receiving the adjustment operations through the operation controls. For example, a corresponding operation control may be set for each type of adjustment operation. Alternatively, the method of receiving adjustment operations may also include receiving the adjustment operations through annotation markers or ultrasound images. For example, a user can click on one of the annotation markers with the mouse and drag it; all annotation markers will then scale or rotate accordingly.

[0103] By scaling, rotating, flipping, and moving the annotation markers, each marker can be effectively aligned with the actual anatomical structure of the four-chamber heart section, thus achieving rapid annotation. It should be noted that the above adjustments are user-executable. Users may need to perform all of these adjustments, or only some; for example, a user might only need to perform a rotation operation to position multiple markers correctly. In some cases, no adjustments may be required, as the initial annotation markers are already in the correct position.

[0104] In summary, the ultrasound image annotation method 200 of this invention can annotate multiple anatomical structures in a four-chamber view ultrasound image at one time, thereby improving the annotation efficiency of ultrasound images.

[0105] Another aspect of this invention provides a method for annotating ultrasound images, such as... Figure 6 As shown, another embodiment of the ultrasound image annotation method 600 of the present invention includes the following steps:

[0106] In step S610, an ultrasound image of the four-chamber view of the target object is acquired and displayed;

[0107] In step S620, in response to the annotation operation, multiple annotation markers characterizing multiple anatomical structures in the four-chamber view are generated, wherein the distance and / or angle of the multiple annotation markers relative to the center of the multiple annotation markers can change synchronously with the adjustment operation.

[0108] In some embodiments, the position of the cruciform structure in the four-chamber view in the ultrasound image, or the position of the intersection of the cruciform structure in the four-chamber view in the ultrasound image, can be determined automatically or manually, and the position can be indicated by a first marker displayed on the ultrasound image. The center of the multiple annotation markers characterizing multiple anatomical structures in the four-chamber view can be based on the position of the cruciform structure, or the multiple annotation markers characterizing multiple anatomical structures in the four-chamber view can be set at a preset position (e.g., the center of the ultrasound image) or a random position, and the position of the multiple anatomical structures can be synchronously adjusted by the user.

[0109] For example, the multiple anatomical structures in the four-chamber view include the left atrium, left ventricle, right atrium, and right ventricle. Correspondingly, multiple annotation markers include left atrial annotation markers, left ventricular annotation markers, right atrial annotation markers, and right ventricular annotation markers, which can be text markers or graphic markers. In some embodiments, the locations of the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image can also be identified, and multiple annotation markers can be generated based on the locations of the left atrium, left ventricle, right atrium, and right ventricle.

[0110] For example, the adjustment operation of the annotation markers includes at least one of scaling, rotation, flipping, and moving operations. The scaling operation is an adjustment operation that fixes the center positions of multiple annotation markers and synchronously adjusts the distance between the multiple annotation markers and their centers; the rotation operation is an adjustment operation that fixes the center positions of multiple annotation markers and synchronously adjusts the angle between the multiple annotation markers and their centers; the flipping operation is an adjustment operation that flips two opposite annotation markers relative to the center of the multiple annotation markers; and the moving operation is an adjustment operation that causes the multiple annotation markers to move synchronously following the adjustment operation of the center positions of the multiple annotation markers.

[0111] More specific details of the ultrasound image annotation method 600 can be found in the relevant description of the ultrasound image annotation method 200, and will not be repeated here. The ultrasound image annotation method 600 of this embodiment can annotate multiple anatomical structures in a four-chamber view ultrasound image at one time, improving the annotation efficiency of ultrasound images.

[0112] This invention also provides an ultrasound imaging system for implementing the ultrasound image annotation method 200 or ultrasound image annotation method 600 described above. Now refer back to... Figure 1 This ultrasound imaging system can achieve the following: Figure 1 The ultrasound imaging system 100 shown may include an ultrasound probe 110, a transmitting circuit 112, a receiving circuit 114, a processor 116, and a display 118. Optionally, the ultrasound imaging system 100 may also include a transmit / receive selection switch 120 and a beamforming module 122. The transmitting circuit 112 and the receiving circuit 114 can be connected to the ultrasound probe 110 through the transmit / receive selection switch 120. The relevant descriptions of each component can be referred to the relevant descriptions above, and will not be repeated here.

[0113] The transmitting circuit 112 is used to excite the ultrasound probe 110 to emit ultrasound waves toward the target tissue; the receiving circuit 112 is used to control the ultrasound probe 110 to receive the echo of the ultrasound waves to obtain an ultrasound echo signal. The processor 116 processes the ultrasound echo signal to obtain an ultrasound image and controls the display 118 to display the ultrasound image obtained by the processor 116. The processor 116 is also used to execute the steps of ultrasound image annotation method 200 or ultrasound image annotation method 600 to generate annotation marks and control the display 118 to display the annotation marks obtained by the processor 116.

[0114] The above only describes the main functions of each component of the ultrasound imaging system. For more details, please refer to the relevant descriptions of ultrasound image annotation methods 200 and 600, which will not be repeated here. The ultrasound imaging system of this embodiment can annotate multiple anatomical structures in a four-chamber view ultrasound image at the same time, improving the annotation efficiency of ultrasound images.

[0115] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention thereto. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0116] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0117] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0118] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0119] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0120] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0121] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0122] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0123] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0124] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for annotating ultrasound images, characterized in that, The method includes: Acquire and display ultrasound images of a four-chamber view of the target object; Based on the ultrasound image or the ultrasound data corresponding to the ultrasound image, determine the position of the cross-shaped structure of the four-chamber view in the ultrasound image; Using the position corresponding to the cross structure as a reference, multiple annotation marks representing multiple anatomical structures in the four-chamber view are generated on the ultrasound image, wherein the center of the multiple annotation marks corresponds to the reference, and the distance and / or angle of the multiple annotation marks relative to the center of the multiple annotation marks can change synchronously with the adjustment operation; The step of determining the position of the cross-shaped structure in the four-chamber view of the ultrasound image based on the ultrasound image or the ultrasound data corresponding to the ultrasound image includes: determining the regions corresponding to the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image based on the ultrasound image of the four-chamber view or the ultrasound data corresponding to the ultrasound image; and determining the position of the cross-shaped structure in the ultrasound image based on the intersection point of the regions corresponding to the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image. Furthermore, determining the position of the cross-shaped structure in the ultrasound image based on the intersection point of the corresponding regions of the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image includes: determining the center point of the corresponding region of the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image respectively; determining the line connecting the center point of the region corresponding to the left atrium and the center point of the region corresponding to the right ventricle, and the line connecting the center point of the region corresponding to the left ventricle and the center point of the region corresponding to the right atrium; and determining the intersection of the two lines as the position of the cross-shaped structure in the ultrasound image.

2. A method for annotating ultrasound images, characterized in that, The method includes: Acquire and display ultrasound images of a four-chamber view of the target object; Based on the ultrasound image or the ultrasound data corresponding to the ultrasound image, determine the position of the cross-shaped structure of the four-chamber view in the ultrasound image; Using the position corresponding to the cross structure as a reference, multiple annotation marks representing multiple anatomical structures in the four-chamber view are generated on the ultrasound image, wherein the center of the multiple annotation marks corresponds to the reference, and the distance and / or angle of the multiple annotation marks relative to the center of the multiple annotation marks can change synchronously with the adjustment operation; The step of determining the position of the cruciate structure of the four-chamber view in the ultrasound image based on the ultrasound image or the ultrasound data corresponding to the ultrasound image includes: The ultrasound images of the four-chamber view or the ultrasound data corresponding to the ultrasound images are identified based on a machine learning model to determine the region corresponding to the cruciate structure. The position of the cross-shaped structure is determined based on the center point of the region corresponding to the cross-shaped structure.

3. The method according to claim 1 or 2, characterized in that, The determination of the position of the cross-shaped structure of the four-chamber view in the ultrasound image based on the ultrasound image or the ultrasound data corresponding to the ultrasound image includes: The location operation is performed based on the ultrasound image reception to locate the position corresponding to the cross structure, and the position corresponding to the cross structure is determined according to the location operation.

4. The method according to claim 1 or 2, characterized in that, The plurality of annotation markers include left atrial annotation markers, left ventricular annotation markers, right atrial annotation markers, and right ventricular annotation markers, wherein the left atrial annotation markers, left ventricular annotation markers, right atrial annotation markers, and right ventricular annotation markers are text markers or graphic markers.

5. The method according to claim 4, characterized in that, The distances between the left atrial annotation mark, left ventricular annotation mark, right atrial annotation mark, and right ventricular annotation mark and the reference are equal, and the line connecting the left atrial annotation mark and the right ventricular annotation mark is perpendicular to the line connecting the left ventricular annotation mark and the right atrial annotation mark.

6. The method according to claim 1 or 2, characterized in that, The adjustment operation includes at least one of the following: The positions of the centers of the plurality of annotation marks on the ultrasound image are fixed, and the distance between the plurality of annotation marks and their centers is adjusted synchronously. The centers of the plurality of annotation marks are fixed in position on the ultrasound image, and the angles of the plurality of annotation marks relative to their centers are adjusted synchronously.

7. The method according to claim 1 or 2, characterized in that, The method further includes: Two opposing annotation markers among the plurality of annotation markers may be flipped relative to the center of the plurality of annotation markers during a flip adjustment operation; and / or, The multiple annotation markers can move synchronously as the center position of the multiple annotation markers is adjusted.

8. The method according to claim 1 or 2, characterized in that, The method further includes: Display operation controls for adjusting the plurality of annotation marks, and receive adjustment operations through the operation controls to adjust the distance and angle of the plurality of annotation marks relative to the reference.

9. A method for annotating ultrasound images, characterized in that, The method includes: Acquire and display ultrasound images of a four-chamber view of the target object; In response to the annotation operation, multiple annotation markers characterizing multiple anatomical structures in the four-chamber view are generated, wherein the distance and / or angle of the multiple annotation markers relative to the center of the multiple annotation markers can change synchronously with the adjustment operation; Based on the ultrasound image or the ultrasound data corresponding to the ultrasound image, determine the position of the cross structure of the four-chamber view in the ultrasound image for the above-mentioned annotation operation; The step of determining the position of the cross-shaped structure in the four-chamber view of the ultrasound image based on the ultrasound image or the ultrasound data corresponding to the ultrasound image includes: determining the regions corresponding to the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image based on the ultrasound image of the four-chamber view or the ultrasound data corresponding to the ultrasound image; and determining the position of the cross-shaped structure in the ultrasound image based on the intersection point of the regions corresponding to the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image. Furthermore, determining the position of the cross-shaped structure in the ultrasound image based on the intersection point of the corresponding regions of the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image includes: determining the center point of the corresponding region of the left atrium, left ventricle, right atrium, and right ventricle in the ultrasound image respectively; determining the line connecting the center point of the region corresponding to the left atrium and the center point of the region corresponding to the right ventricle, and the line connecting the center point of the region corresponding to the left ventricle and the center point of the region corresponding to the right atrium; and determining the intersection of the two lines as the position of the cross-shaped structure in the ultrasound image.

10. The method according to claim 9, characterized in that, The plurality of annotation markers include left atrial annotation markers, left ventricular annotation markers, right atrial annotation markers, and right ventricular annotation markers, wherein the left atrial annotation markers, left ventricular annotation markers, right atrial annotation markers, and right ventricular annotation markers are text markers or graphic markers.

11. The method according to claim 9, characterized in that, The method further includes: indicating the position of the intersection of the cross structure of the four-chamber view in the ultrasound image by a first identifier displayed on the ultrasound image.

12. The method according to claim 9, characterized in that, The method further includes: determining the position of the intersection of the cross-shaped structure of the four-chamber view in the ultrasound image, and indicating the position of the intersection of the cross-shaped structure of the four-chamber view in the ultrasound image by a first identifier displayed on the ultrasound image.

13. The method according to claim 9, characterized in that, The adjustment operation includes at least one of the following: The center positions of the multiple annotation marks are fixed, and the distance between the multiple annotation marks and their centers is adjusted synchronously. The center positions of the multiple annotation markers are fixed, and the angles of the multiple annotation markers relative to their centers are adjusted synchronously.

14. The method according to claim 9, characterized in that, The method further includes: Two opposing annotation markers among the plurality of annotation markers may be flipped relative to the center of the plurality of annotation markers during a flip adjustment operation; and / or, The multiple annotation markers can move synchronously as the center position of the multiple annotation markers is adjusted.

15. An ultrasound imaging system, characterized in that, The ultrasound imaging system includes: Ultrasonic probe; A transmitting circuit is used to excite the ultrasound probe to emit ultrasound waves toward the heart of the target object; A receiving circuit is used to control the ultrasonic probe to receive the echo of the ultrasonic wave and obtain the echo signal; A processor is configured to generate an ultrasound image of a four-chamber view of a target object based on the echo signal, and to perform an annotation method for the ultrasound image as described in any one of claims 1-14. A display for showing the ultrasound images.

Images