Ultrasound imaging method, ultrasound imaging system, and non-transitory computer readable medium

Abstract

The invention provides an ultrasonic imaging method. The ultrasonic imaging method comprises the steps that an ultrasonic image of a to-be-imaged tissue is generated and displayed; detecting the sight line of a user, and automatically determining a region of interest of the user on the ultrasonic image; and optimizing the ultrasonic imaging parameters of the region of interest, generating an optimized ultrasonic image for the region of interest, and displaying the optimized ultrasonic image. The other embodiment of the invention further provides an ultrasonic imaging system and a non-transient computer readable medium.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging, and in particular to an ultrasonic imaging method, an ultrasonic imaging system and a non-transient computer-readable medium for automatically optimizing ultrasonic imaging parameters. Background Art

[0002] Ultrasonic imaging technology is a real-time, non-destructive imaging technology. It uses an ultrasonic probe to send ultrasonic signals to the part to be imaged and receive echo signals, thereby obtaining ultrasonic data about the part to be imaged. After processing, the ultrasonic data can be used to obtain an ultrasonic image of the part to be imaged.

[0003] During the imaging process, the user needs to adjust the ultrasound imaging parameters. There may be many reasons for this. For example, the optimal imaging parameters for different parts to be imaged are different. Even different positions of the same part to be imaged have different optimal imaging parameters. During the actual ultrasound scanning process, the user needs to adjust the ultrasound imaging parameters by operating the user interface to ensure that the ultrasound image they need has satisfactory image quality. However, the adjustment of ultrasound imaging parameters is difficult. On the one hand, different imaging positions have different optimal imaging parameters, which depends on the user's experience and is more difficult for novice doctors. On the other hand, there are multiple ultrasound imaging parameters that affect the image quality, and the adjustment is also more complicated. The above process also relies heavily on manual labor, so the work is cumbersome and time-consuming. Summary of the invention

[0004] The above-mentioned deficiencies, disadvantages and problems are addressed herein and will be understood by reading and understanding the following description.

[0005] In some embodiments of the present application, an ultrasound imaging method is provided, including: generating and displaying an ultrasound image of a tissue to be imaged; detecting a user's line of sight and automatically determining an area of ​​interest of the user on the ultrasound image; and optimizing ultrasound imaging parameters of the area of ​​interest, generating and displaying an optimized ultrasound image for the area of ​​interest.

[0006] In some other embodiments of the present application, an ultrasound imaging system is provided, comprising: a probe, the probe is used to collect ultrasound data about the tissue to be imaged; a processor; a display, the display receives a signal from the processor and displays it; and an image acquisition unit, which is used to acquire an image of the user and send it to the processor. The processor is configured to perform the following method: generate and display an ultrasound image of the tissue to be imaged; detect the user's line of sight, automatically determine the user's area of ​​interest on the ultrasound image; and optimize the ultrasound imaging parameters of the area of ​​interest, generate and display an optimized ultrasound image for the area of ​​interest.

[0007] In some embodiments of the present application, a non-transitory computer-readable medium is also provided, wherein the non-transitory computer-readable medium stores a computer program, wherein the computer program has at least one code segment, and the at least one code segment can be executed by a machine so that the machine performs the steps of the following method: generating and displaying an ultrasound image of the tissue to be imaged; detecting the user's line of sight and automatically determining the user's area of ​​interest on the ultrasound image; and optimizing the ultrasound imaging parameters of the area of ​​interest, generating and displaying an optimized ultrasound image for the area of ​​interest.

[0008] It should be understood that the brief description above is provided to introduce some concepts further described in the detailed description in a simplified form. It is not meant to identify key or essential features of the claimed subject matter, the scope of which is solely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages mentioned above or in any section of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present application will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0010] Figure 1 is a schematic diagram 100 of an ultrasound imaging system according to some embodiments of the present application;

[0011] Figure 2 is a flow chart of an ultrasound imaging method 200 in some embodiments of the present application;

[0012] Figure 3 is a schematic diagram 300 of determining a user's area of ​​interest on an ultrasound image in some embodiments of the present application;

[0013] Figure 4 A schematic diagram 400 showing optimization of ultrasound images in some embodiments of the present application is shown;

[0014] Figure 5 is a schematic diagram 500 of optimizing a volume ultrasound image in some other embodiments of the present application;

[0015] Figure 6 is a flowchart of an ultrasound imaging method 600 in some other embodiments of the present application. DETAILED DESCRIPTION

[0016] The specific embodiments of the present invention will be described below. It should be noted that in the specific description of these embodiments, in order to be concise, it is impossible for the present invention to provide a detailed description of all the features of the actual embodiments. It should be understood that in the actual implementation of any embodiment, just as in the process of any engineering project or design project, in order to achieve the specific goals of the developer and to meet system-related or business-related restrictions, various specific decisions are often made, and this will also change from one embodiment to another. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the content disclosed by the present invention, some changes such as design, manufacturing or production based on the technical content disclosed in the present disclosure are just conventional technical means, and should not be understood as insufficient content of the present disclosure.

[0017] Unless otherwise defined, the technical or scientific terms used in the claims and the specification shall have the usual meanings understood by persons of ordinary skill in the relevant technical field. The words "first", "second" and similar terms used in the present invention and the claims do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "one" or "an" do not indicate a quantitative limitation, but indicate the existence of at least one. Words such as "include" or "comprise" and similar terms mean that the elements or objects appearing before "include" or "comprise" include the elements or objects listed after "include" or "comprise" and their equivalent elements, and do not exclude other elements or objects. Words such as "connect" or "connected" and similar terms are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0018] Figure 1 A schematic block diagram of an embodiment of an ultrasound imaging system 100 is shown. The ultrasound imaging system 100 may include a controller circuit 102 operably connected to a communication circuit 104, a display 138, an image acquisition unit 144, a user interface 142, a probe 126, and a memory 106.

[0019] The controller circuit 102 is configured to control the operation of the ultrasound imaging system 100. The controller circuit 102 may include one or more processors. Optionally, the controller circuit 102 may include a central processing unit (CPU), one or more microprocessors, a graphics processing unit (GPU), or any other electronic component capable of processing input data according to specific logic instructions. Optionally, the controller circuit 102 may include and / or represent one or more hardware circuits or circuit systems, which include, connect, or include and connect one or more processors, controllers, and / or other hardware logic-based devices. Additionally or alternatively, the controller circuit 102 may execute instructions stored on a tangible and non-transitory computer-readable medium (e.g., memory 106).

[0020] The controller circuit 102 may be operably connected to and / or control the communication circuit 104. The communication circuit 104 is configured to receive and / or transmit information along a bidirectional communication link with one or more optional ultrasound imaging systems, remote servers, etc. The remote server may represent a remotely stored medical image including patient information, machine learning algorithms, previous scans and / or treatments of the patient, etc. The communication circuit 104 may represent hardware for transmitting and / or receiving data along a bidirectional communication link. The communication circuit 104 may include a transceiver, a receiver, a transceiver, etc., and associated circuit systems (e.g., antennas) for communicating (e.g., transmitting and / or receiving) in a wired and / or wireless manner with the one or more optional ultrasound imaging systems, remote servers, etc. For example, protocol firmware for transmitting and / or receiving data along a bidirectional communication link may be stored in a memory 106 accessed by the controller circuit 102. The protocol firmware provides the controller circuit 102 with a network protocol grammar to assemble data packets, build and / or segment data received along a bidirectional communication link, etc.

[0021] The bidirectional communication link may be a wired (e.g., via physical conductors) and / or wireless communication (e.g., utilizing radio frequency (RF)) link for exchanging data (e.g., data packets) between the one or more alternative ultrasound imaging systems, remote servers, etc. The bidirectional communication link may be based on a standard communication protocol, such as Ethernet, TCP / IP, WiFi, 802.11, a custom communication protocol, Bluetooth, etc.

[0022] The controller circuit 102 is operably connected to a display 138 and a user interface 142. The display 138 may include one or more liquid crystal displays (e.g., light emitting diode (LED) backlights), organic light emitting diode (OLED) displays, plasma displays, CRT displays, etc. The display 138 may display patient information, one or more medical images and / or videos, a graphical user interface or component, one or more 2D, 3D, or 4D ultrasound image data sets from ultrasound data stored in the memory 106, or currently acquired anatomical measurements, diagnoses, processing information, etc., received by the display 138 from the controller circuit 102.

[0023] The user interface 142 controls the operation of the controller circuit 102 and the ultrasound imaging system 100. The user interface 142 is configured to receive input from a clinician and / or an operator of the ultrasound imaging system 100. The user interface 142 may include a keyboard, a mouse, a touchpad, a trackball, one or more physical buttons, and the like. Optionally, the display 138 may be a touch screen display that includes at least a portion of the user interface 142. For example, a portion of the user interface 142 may correspond to a graphical user interface (GUI) generated by the controller circuit 102 and displayed on the display 138. The touch screen display may detect the presence of a touch from an operator on the display 138, and may also identify the location of the touch relative to the surface area of ​​the display 138. For example, a user may select one or more user interface components of the user interface (GUI) shown on the display by touching or contacting the display 138. The user interface components may correspond to icons, text boxes, menu bars, and the like shown on the display 138. The user interface components may be selected, manipulated, utilized, interacted with, and the like by a clinician to instruct the controller circuit 102 to perform one or more operations as described herein. For example, the touch may be applied by at least one of a person's hand, a glove, a stylus, and the like.

[0024] The memory 106 includes parameters, algorithms, one or more ultrasound examination protocols, data values, etc. used by the controller circuit 102 to perform one or more operations described in the present application. The memory 106 can be a tangible and non-transitory computer-readable medium, such as a flash memory, RAM, ROM, EEPROM, etc. The memory 106 may include a set of learning algorithms (e.g., convolutional neural network algorithms, deep learning algorithms, decision tree learning algorithms, etc.) configured to define image analysis algorithms. When executing the image analysis algorithm, the controller circuit 102 is configured to identify anatomical features contained in the ultrasound image. Optionally, the image analysis algorithm can be received by the communication circuit 104 along one of the two-way communication links and stored in the memory 106. The types of image analysis algorithms can be diverse, and an exemplary description is given below.

[0025] The image analysis algorithm may be defined by one or more algorithms to identify one or more anatomical features in an ultrasound image (e.g., boundaries, thicknesses, changes in pixel values, valves, cavities, chambers, edges or inner layers, vascular structures, etc.). The one or more anatomical features may represent features of pixels and / or voxels of a medical image, such as histogram oriented gradients, point features, covariance features, binary pattern features, etc. For example, the image analysis algorithm may be defined using predictions of object recognition within a medical image using one or more deep neural networks.

[0026] The image analysis algorithm may correspond to an artificial neural network formed by the controller circuit 102 and / or the remote server. The image analysis algorithm may be divided into two or more layers, such as an input layer that receives an input image, an output layer that outputs an output image, and / or one or more intermediate layers. The layers of the neural network represent different groups or sets of artificial neurons, which may represent different functions that are executed by the controller circuit 102 on an input image (e.g., an ultrasound image acquired and / or generated by the ultrasound imaging system 100) to identify an object of the input image and determine one or more anatomical features contained in the input image. The artificial neurons in the layers of the neural network may examine each pixel in the input image. The artificial neurons apply different weights in the function applied to the input image in an attempt to identify the object in the input image. The neural network generates an output image by assigning or associating different pixels in the output image with different anatomical features based on an analysis of the pixel characteristics.

[0027] The image analysis algorithm is defined by a plurality of training images, which may be grouped into different anatomical planes of interest of an anatomical structure of interest. The training images may represent different orientations and / or cross-sections of the anatomical structure of interest corresponding to different fields of view. Additionally or alternatively, the image analysis algorithm may be defined by the controller circuit based on a classification model. The classification model may correspond to a machine learning algorithm based on a classifier (e.g., a random forest classifier, principal component analysis, etc.) that is configured to identify and / or assign anatomical features to multiple categories or classes based on overall shape, spatial position relative to the anatomical structure of interest, intensity, etc.

[0028] Based on the relationship of the anatomical features to each other, the modality, etc., the controller circuit 102 executing an image analysis algorithm (eg, an image analysis algorithm) may determine the anatomical features contained in the current ultrasound image.

[0029] Additionally or alternatively, the controller circuit 102 may define separate image analysis algorithms that are customized and / or configured for different selected anatomical structures of interest. For example, a plurality of image analysis algorithms may be stored in the memory 106. Each of the plurality of image analysis algorithms may be customized and / or configured based on different training images (e.g., a set of input images) to configure layers of different neural networks to select anatomical structures of interest, classification models, supervised learning models, and the like.

[0030] In some other embodiments, the image analysis algorithm of the controller circuit 102 can also be any other method in the art. For example, it can be an algorithm that identifies the boundary of the structure of interest in the image based on the change of the gray value of the image. For example, a certain gray value threshold can be set, and the threshold can be used to compare adjacent pixels in the image. Once the gray value of the adjacent pixel is greater than the above gray value threshold, it is determined that this is the position to be identified, for example, it can be the boundary of a specific anatomical feature. In addition, it should be noted that the above embodiments of the present application describe an example of the controller circuit 102 performing image recognition, but the recognition method is not limited to this and will not be exhaustively listed here. The specific application method of the recognition result of the anatomical feature will be described exemplarily below.

[0031] Continue to refer Figure 1 , the ultrasound imaging system 100 may include a probe 126 having a transmitter 122, a transmit beamformer 121, and a detector / SAP electronics 110. The detector / SAP electronics 110 may be used to control the switching of the transducer elements 124. The detector / SAP electronics 110 may also be used to group the transducer elements 124 into one or more sub-apertures.

[0032] The probe 126 may be configured to acquire ultrasound data or information from tissue to be imaged (e.g., a fetus, an organ, a blood vessel, a heart, a bone, etc.). The probe 126 is communicatively connected to the controller circuit 102 via a transmitter 122. The transmitter 122 transmits a signal to the transmit beamformer 121 based on the acquisition settings received by the controller circuit 102. The acquisition settings may define the amplitude, pulse width, frequency, gain setting, scanning angle, power, time gain compensation (TGC), resolution, etc. of the ultrasound pulses emitted by the transducer elements 124. The transducer elements 124 transmit pulsed ultrasound signals into a patient (e.g., a body). The acquisition settings may be defined by a user operating the user interface 142. The signal transmitted by the transmitter 122 then drives a plurality of transducer elements 124 within the transducer array 112.

[0033] The transducer element 124 transmits a pulsed ultrasound signal into a body (e.g., a patient) or volume corresponding to an acquisition setting along one or more scanning planes. The ultrasound signal may include, for example, one or more reference pulses, one or more push pulses (e.g., shear waves), and / or one or more pulsed wave Doppler pulses. At least a portion of the pulsed ultrasound signal is backscattered from the tissue to be imaged (e.g., organs, bones, heart, breast tissue, liver tissue, cardiac tissue, prostate tissue, neonatal brain, embryo, abdomen, etc.) to produce an echo. Depending on the depth or movement, the echo is delayed in time and / or frequency and received by the transducer element 124 within the transducer array 112. The ultrasound signal can be used for imaging, for generating and / or tracking shear waves, for measuring changes in position or velocity within an anatomical structure, differences in compression displacement of tissue (e.g., strain), and / or for treatment and other purposes. For example, the probe 126 can deliver low energy pulses during imaging and tracking, deliver medium and high energy pulses to generate shear waves, and deliver high energy pulses during treatment.

[0034] The transducer element 124 converts the received echo signal into an electrical signal that can be received by the receiver 128. The receiver 128 may include one or more amplifiers, analog / digital converters (ADCs), etc. The receiver 128 may be configured to amplify the received echo signal after appropriate gain compensation, and convert the analog signals received from each transducer element 124 into digitized signals sampled uniformly in time. The digitized signals representing the received echo are temporarily stored in the memory 106. The digitized signals correspond to the backscattered waves received by each transducer element 124 at different times. After digitization, the signal can still retain the amplitude, frequency, and phase information of the backscattered waves.

[0035] Alternatively, the controller circuit 102 may retrieve the digitized signal stored in the memory 106 for use with the beamformer processor 130. For example, the controller circuit 102 may convert the digitized signal to a baseband signal or compress the digitized signal.

[0036] The beamformer processor 130 may include one or more processors. Optionally, the beamformer processor 130 may include a central processing unit (CPU), one or more microprocessors, or any other electronic component capable of processing input data according to specific logic instructions. Additionally or alternatively, the beamformer processor 130 may execute instructions stored on a tangible and non-transitory computer-readable medium (e.g., memory 106) to perform beamforming calculations using any suitable beamforming method, such as adaptive beamforming, synthetic transmit focusing, aberration correction, synthetic aperture, clutter suppression, and / or adaptive noise control, etc. Optionally, the beamformer processor 130 may be integrated with and / or be part of the controller circuit 102. For example, operations described as being performed by the beamformer processor 130 may be configured to be performed by the controller circuit 102.

[0037] The beamformer processor 130 performs beamforming on the digitized signals of the transducer elements and outputs a radio frequency (RF) signal. The RF signal is then provided to an RF processor 132 that processes the RF signal. The RF processor 132 may include one or more processors. As desired, the RF processor 132 may include a central processing unit (CPU), one or more microprocessors, or any other electronic component capable of processing input data according to specific logic instructions. Additionally or alternatively, the RF processor 132 may execute instructions stored on a tangible and non-transitory computer-readable medium (e.g., memory 106). Optionally, the RF processor 132 may be integrated with the controller circuit 102 and / or as part of the controller circuit. For example, operations described as being performed by the RF processor 132 may be configured to be performed by the controller circuit 102.

[0038] The RF processor 132 may generate different ultrasound image data types and / or modes for multiple scan planes or different scan modes based on the predetermined settings of the first model, such as B-mode, color Doppler (e.g., color blood flow, velocity / power / variance), tissue Doppler (velocity), and Doppler energy. For example, the RF processor 132 may generate tissue Doppler data for multiple scan planes. The RF processor 132 collects information related to multiple data slices (e.g., I / Q, B-mode, color Doppler, tissue Doppler, and Doppler energy information) and stores the data information in the memory 106, which may include time stamps and orientation / rotation information.

[0039] Optionally, the RF processor 132 may include a complex demodulator (not shown) that demodulates the RF signal to form an IQ data pair representing the echo signal. The RF or IQ signal data may then be provided directly to the memory 106 for storage (e.g., temporary storage). Optionally, the output of the beamformer processor 130 may be communicated directly to the controller circuit 102.

[0040] The controller circuit 102 may be configured to process the acquired ultrasound data (e.g., RF signal data or IQ data pairs) and prepare and / or generate ultrasound image data frames representing anatomical structures of interest for display on the display 138. The acquired ultrasound data may be processed by the controller circuit 102 in real time as echo signals are received during a scanning or treatment process for an ultrasound examination. Additionally or alternatively, the ultrasound data may be temporarily stored in the memory 106 during the scanning process and processed in a less real-time manner in an on-the-go or off-line operation. The memory 106 may be used to store processed frames of acquired ultrasound data that are not intended for immediate display, or to store post-processed images (e.g., shear wave images, strain images), firmware or software corresponding to, for example, a graphical user interface, one or more default image display settings, programmed instructions, and the like.

[0041] Furthermore, the ultrasonic imaging system 100 may also include an image acquisition unit 144. The image acquisition unit 144 communicates with the controller circuit 102. The image acquisition unit 144 can acquire an environmental image and transmit the image data to the controller circuit 102. The processor of the controller circuit 102 can analyze and process the environmental image. In one example, the processor can identify the user in the image and the direction of the user's gaze through the acquired environmental image. The specific identification method can be any in the prior art and is not limited here.

[0042] The image acquisition unit 144 can be any device with an image acquisition function, such as a camera, a 3D camera, a video camera, etc. In an optional embodiment, the image acquisition unit 144 can be integrated and fixedly arranged on the ultrasonic imaging system 100. For example, it can be arranged on the display 138. Such a setting method can, on the one hand, ensure that the relative positions of the display 138 and the image acquisition unit 144 are fixed, thereby ensuring the accuracy of the algorithm for analyzing the direction in which the user is looking at the display 138 on the ultrasonic imaging system 100. In another example, the image acquisition unit 144 can also be an external device, and its installation position can be adjusted and calibrated. In addition, the image acquisition unit 144 can include multiple, and the simultaneous operation of multiple image acquisition units can further ensure that the image data transmitted to the controller circuit 102 is more comprehensive and the analysis results are more accurate. The purpose and advantages of analyzing the above image data will be described in detail below.

[0043] When the user uses the ultrasound imaging system 100 for ultrasound scanning, the user usually moves the probe to the corresponding position of the tissue to be imaged to obtain ultrasound data about the tissue to be imaged. In addition, after reaching the corresponding position, the parameters of ultrasound imaging must be adjusted to obtain the optimal ultrasound image of the tissue to be imaged. Among them, ultrasound imaging parameters include many types. As listed above, it can include parameters related to ultrasound data acquisition during ultrasound imaging, such as the amplitude, pulse width, frequency, gain setting, scanning angle, power, time gain compensation (TGC), resolution, frame frequency, etc. of the ultrasound pulse. It can also be a parameter related to image processing during ultrasound imaging, such as adaptive beamforming, synthetic transmit focusing, aberration correction, synthetic aperture, clutter suppression and / or adaptive noise control and other beamforming related parameters. In the same tissue to be imaged, different anatomical features of interest may be suitable for different ultrasound imaging parameters, for example, different depths or different types of anatomical features are suitable for different imaging parameters. Alternatively, even if certain imaging parameters (e.g., frame frequency, resolution, power, etc.) are applicable to improving the image quality of all positions in the same ultrasound image, considering the safety issues caused by the power upper limit, heat generation, etc. of the probe, the image quality needs to be sacrificed in the actual ultrasound imaging process to ensure the safety and stability of the imaging process. At least for the above reasons, during the ultrasound imaging process, the user needs to make a comprehensive judgment on the current ultrasound imaging based on experience, determine the optimal ultrasound scanning parameters, and make manual adjustments. This consumes a lot of manpower and time on the one hand, and is a huge challenge for inexperienced users on the other. At least because of the existence of the above problems, improvements are proposed in some embodiments of the present application.

[0044] refer to Figure 2 , showing a schematic diagram of an ultrasound imaging method 200 in some embodiments of the present application.

[0045] In step 201, an ultrasound image of the tissue to be imaged is generated and displayed. This process can be implemented by a processor. Specifically, the processor can receive ultrasound data of the tissue to be imaged acquired by the probe, and process the data to generate an ultrasound image. Further, the processor also displays the generated ultrasound image so that it can be observed.

[0046] In step 203, the user's line of sight is detected, and the user's area of ​​interest on the ultrasound image is automatically determined. Specifically, the processor can obtain an image of the current environment and perform image analysis. Image analysis can determine the user in the environment image and the direction of the user's line of sight. The above-mentioned image analysis algorithm and process can be any in the prior art. For example, by any known method, the face information and eye information in the environment image are identified, and the direction of the eye is comprehensively judged, so as to determine which area of ​​the ultrasound image the user is looking at.

[0047] In step 205, the ultrasonic imaging parameters of the region of interest are optimized, and an optimized ultrasonic image for the region of interest is generated and displayed. The ultrasonic imaging parameters refer to parameters related to generating ultrasonic images. They may be one or more of those listed above in the present application, or other parameters related to ultrasonic imaging in the art. After determining the region of interest, the processor may automatically optimize the configuration of imaging parameters for the region of interest, thereby improving the image quality of the region of interest.

[0048] Such a configuration method can automatically determine the area where the user needs to optimize the ultrasonic imaging parameters and automatically optimize the ultrasonic image of the area. In the embodiment of the present application, the inventor realizes that the area where the user wants to optimize the image quality is usually highly consistent with the area where the user's line of sight focuses. On this basis, the position of the user's line of sight on the ultrasonic image is automatically analyzed and judged, that is, the area where the ultrasonic imaging parameters need to be optimized can be determined. The method of the present application, on the one hand, does not require the user to select a specific area to be optimized, so that the workflow is saved and the user's additional operation steps are omitted. On the other hand, the user is not required to perform manual parameter adjustment after selecting a specific area to be optimized. Manual parameter adjustment is not only very cumbersome, but also for inexperienced novices, it is a considerable challenge to adjust many configuration parameters to make the area to be optimized achieve the ideal ultrasonic image quality. Using the method of the present application, the workflow is saved, and the image quality of the area of ​​concern can be quickly improved.

[0049] As described above in this application, ultrasound imaging parameters include various types. In the field of ultrasound imaging, some ultrasound imaging parameters may affect the quality improvement of the entire image. For example, the power of the ultrasound probe, the frame frequency when collecting ultrasound data, etc. will affect the clarity of the entire ultrasound image, or time gain compensation will affect the display effect of the overall ultrasound image. Other ultrasound imaging parameters may improve the quality of ultrasound images in some areas and reduce the quality of ultrasound images in other areas, such as adjustment of the focus position, depth adjustment, etc.

[0050] In an embodiment of the present application, optimizing the ultrasonic imaging parameters of the region of interest includes: optimizing the ultrasonic imaging parameters of at least part of the region of interest, and not optimizing the imaging parameters of at least part of the ultrasonic image outside the region of interest. In other words, the determination of the region of interest and the optimization of the imaging parameters of the region of interest in the present application are carried out in a targeted manner on the region of interest, not simply on the entire ultrasonic image. Such a configuration method includes at least the following advantages: First, some ultrasonic imaging parameters cannot improve the image quality of the entire ultrasonic image. In the present application, the ultrasonic imaging parameters are optimized for the region of interest, which can ensure that the adjustment of the ultrasonic imaging parameters is conducive to improving the image quality of the region of interest. In addition, even for some ultrasonic imaging parameters that can improve the overall ultrasonic image quality, it may bring about adverse effects such as consuming too much processor resources and generating a lot of heat. Dedicating these ultrasonic imaging parameters to the region of interest can save resources as much as possible, avoid adverse effects such as overheating, and further improve the image quality of the region of interest.

[0051] In addition, in the embodiments of the present application, it is also allowed that the optimization area of ​​the ultrasound imaging parameters has a certain deviation relative to the area of ​​interest. For example, optimizing the ultrasound imaging parameters of at least part of the area of ​​interest means that it is allowed that some areas in the area of ​​interest may not be fully optimized, which may depend on factors such as the size of the area of ​​interest and whether the adjusted parameters can meet the needs of the entire area. Not optimizing the imaging parameters of at least part of the ultrasound image outside the area of ​​interest means that a part of the ultrasound image outside the area of ​​interest may also be optimized. This is because, on the one hand, the embodiments of the present application allow a certain deviation in optimization, and on the other hand, it is determined by the ultrasound imaging method itself. For example, in the process of volume imaging using a mechanical 4D probe, the ultrasonic transducer performs a sector-shaped scan in the probe housing. When it is necessary to optimize the ultrasound imaging parameters of the area of ​​interest, such as increasing the frame rate of the area, it is inevitable to optimize the line of sight of the ultrasound image in the entire depth direction including the area of ​​interest. This situation should be allowed.

[0052] For a clearer description of the technical solution of this application, please refer to Figure 3 , showing a schematic diagram 300 of determining a user's area of ​​interest on an ultrasound image in some embodiments of the present application.

[0053] like Figure 3As shown, the image acquisition unit 302 is integrated on the display 301. In some embodiments, the image acquisition unit 302 can be fixedly set on the display 301. In this way, the degree of device integration is increased. Moreover, no matter how the position of the display 301 relative to the user changes, the relative position of the display 301 and the eyes, and the relative position of the image acquisition unit 302 and the eyes, can always maintain a good correspondence, thereby improving the accuracy of judging the user's focus area. In addition, although Figure 3 The number of the image acquisition unit 302 shown is one, but in other embodiments, the image acquisition unit 302 may include multiple. Multiple image acquisition units simultaneously capture images of the user's eyes, which can provide more abundant information for determining the user's focus area and ensure more accurate determination.

[0054] The image acquisition unit 302 is used to acquire image information of the surrounding environment. For example, the image acquisition unit 302 can capture images within a certain range in front of the display 301. Usually, the user 303 needs to be at a certain distance in front of the display 301 to observe the ultrasound image 311 on the display 301. At this time, the image acquisition unit 302 can capture an image including the user 303 and the user's 303 eyes 331, and transmit it to the processor (not shown). In some examples, the direction of the user's line of sight 332 can be determined by a pre-stored algorithm. For example, the direction of the line of sight 332 can be determined by various information such as the direction of the center of the eye and the position of the pupil. The direction of the line of sight 332 can be understood as the direction (or direction and distance) of the line of sight 332 relative to the image acquisition unit 302. The relative position, direction and other information between the image acquisition unit 302 and the display 301 are set in advance. At this time, the user's focus point 333 on the ultrasound image 311 can be determined based on the line of sight direction 332.

[0055] It can be understood that the way of sight detection can be diverse. In one embodiment, detecting the sight of the user includes detecting the position of the user's sight towards the ultrasound image. That is, while the image acquisition unit 302 acquires the environmental image in real time, the algorithm detects the user's focus position on the ultrasound image in real time. Once the focus position is determined, the focus area is used to optimize the ultrasound image. In this way, the response speed of the device can be improved and the image quality of the user's focus area can be improved in a timely manner. In another embodiment, detecting the sight of the user includes detecting the position of the user's sight towards the ultrasound image and the duration at the position. That is, in addition to determining the user's focus position on the ultrasound image, the time for which the position is continuously focused is also determined. For example, a time threshold can be set, and only when the user continues to focus on a position of an ultrasound image for more than a time threshold (for example, 3 seconds), the position is regarded as a focus area and the ultrasound imaging is optimized. Such a configuration method can reduce the amount of calculation of the device while avoiding misprocessing of ultrasound imaging parameters. It is particularly suitable for the use scenario of users quickly browsing ultrasound images. It is understood that the above different user gaze detection modes can be automatically set by the machine as a default setting, or can be selected by the user. For example, the user can choose to switch between different modes, or adjust the time threshold.

[0056] Continue to refer Figure 3 By detecting the user's line of sight 332, the user's focus area 312 on the ultrasound image 311 can be automatically determined. In one example, the automatic determination of the focus area 312 can be by: determining the user's focus point 333 on the ultrasound image 311 based on the detection of the user's line of sight 332; and automatically determining a certain range of the ultrasound image around the focus point 333 as the focus area 312.

[0057] In this way, the position where the ultrasound imaging parameters need to be optimized and adjusted can be quickly determined without the need for additional complex data processing. The shape and size of the region of interest 312 can be adjusted according to various situations. For example, it can be selected according to various factors such as the size of the display 301 or the ultrasound image 311. In addition, in some embodiments, the above shape and size can also be adjusted by the user.

[0058] In another example, the automatic determination of the region of interest may be by: determining the user's focus point on the ultrasound image based on the detection of the user's line of sight; performing image recognition on the ultrasound image of the focus point to determine the corresponding anatomical feature of the focus point; and automatically determining the entire ultrasound image of the corresponding anatomical feature as the region of interest. The above-mentioned image recognition method may be any of the prior art, or may be implemented by the embodiments described above in this application, and will not be described in detail here.

[0059] This method can accurately determine the anatomical features that the user is currently interested in, which is more in line with clinical usage needs. Figure 4 Give an example. Figure 4 A schematic diagram 400 of optimizing an ultrasound image in some embodiments of the present application is shown. In Figure 401, the user's focus (not shown) falls on a part of the mitral valve 412 (anatomical feature). At this time, image recognition technology can be used to determine that the anatomical feature corresponding to the focus point is the mitral valve 412. Further, the entire ultrasound image of the mitral valve can be automatically used as the area of ​​interest 411. At this time, when the user images and observes the mitral valve 412, a high-quality image 402 of the entire mitral valve 422 can be obtained. For example, the mitral valve 422 in the image 402 has higher brightness, resolution, etc. than the mitral valve 412 in the unoptimized image 401, which is more conducive to user observation.

[0060] The mitral valve is used as an example of an anatomical feature. In fact, anatomical features can also be of any other type. Anatomical features can be understood as local organs, tissues or lesions with clinical significance. In addition, it should be noted that Figure 3 Other components of the specific ultrasound imaging system, such as a probe, a host, a processor, etc., are not shown. Unless otherwise specified, the ultrasound imaging system and method thereof of any embodiment of the present application are applicable to each other and can be combined.

[0061] By identifying the anatomical features corresponding to the focus point, it can not only be used to assist in determining the focus area, but in some embodiments, it can also further help optimize the ultrasound imaging parameters. Specifically, based on the identification of the corresponding anatomical features, optimizing the ultrasound imaging parameters of the focus area includes configuring the ultrasound imaging parameters according to the corresponding anatomical features. The following is a detailed description through some specific embodiments.

[0062] In some embodiments, in response to the corresponding anatomical feature being smaller in size in the current ultrasound image, the ultrasound image of the region of interest may be automatically enlarged for display. Figure 4A detailed description is given. First, in image 401, through line of sight detection and image recognition, it is determined that the mitral valve 412 is included in the area of ​​interest 411. Further, the anatomical feature of the mitral valve 412 can be specifically optimized to obtain an optimized image 402. Considering that the mitral valve 412 is relatively small in size in the current ultrasound image, the mitral valve 422 is enlarged and displayed in image 402. At this time, the doctor can observe the mitral valve 422 more clearly. This embodiment is also applicable to other anatomical features of small size. For example, capillaries, etc. In addition, image 402 shows a situation where the enlarged image of the mitral valve 422 is overlaid on image 402. In another embodiment, it is also possible not to overlay, but to deform the image 402 so that the entire ultrasound image is still displayed while the area of ​​interest is enlarged.

[0063] In another example, in response to the ultrasound image being a volume image, an optimized viewing angle for the corresponding anatomical feature may be automatically displayed. Figure 5 Provide detailed explanation. Figure 5 A schematic diagram 500 of optimizing a volume ultrasound image in some other embodiments of the present application is shown. First, in image 501, through line of sight detection and image recognition, it is determined that the anatomical feature of the fetal eye 512 is included in the area of ​​interest 511. However, as shown in image 501, the image viewing angle of the fetal eye 512 is blocked in image 501 and cannot be fully observed. In conventional practice, the user needs to move the position of the probe or adjust it by operating the user interface to obtain a satisfactory viewing angle. In this embodiment, after identifying that the area of ​​interest 511 contains the anatomical feature of the fetal eye 512, the optimized viewing angle can be automatically displayed for the anatomical feature, thereby facilitating observation. For example, as shown in image 502, the fetal eye 522 has been adjusted to have a viewing angle that is more conducive to user observation.

[0064] The above two examples are exemplary illustrations of targeted optimization of ultrasound imaging parameters for identified anatomical features. Based on the determination of the type of anatomical features, automatic optimization can also be in other ways. In addition, it should be noted that the optimization methods described in the above embodiments are not opposed to the optimization of any other ultrasound imaging parameters described above in this application. In some embodiments, multiple optimization methods for ultrasound imaging parameters can be selected at the same time. For example, the optimization of general ultrasound imaging parameters such as magnified display of anatomical features and increased resolution and probe power can be selected at the same time (for example, Figure 4 In the example above, the mitral valve was magnified and other imaging parameters were optimized).

[0065] The ultrasound imaging process is a real-time, dynamic process. In order to further improve the work efficiency and user experience in the real-time imaging process, some embodiments of the present application also propose improvements. Figure 6 , showing a flowchart of an ultrasound imaging method 600 in some embodiments of the present application.

[0066] In step 601, the line of sight of the user is continuously detected to automatically determine changes in the area of ​​interest. The manner of detecting the line of sight of the user and the method of automatically determining the area of ​​interest may be the ones described in any of the above embodiments of the present application and will not be described in detail herein. The user needs to observe different ultrasound section images during the ultrasound scanning process, and even for the same image, different positions may be observed. Therefore, the area of ​​interest may be constantly changing. In the steps of the present application, the processor continuously detects and analyzes the line of sight to determine whether the area of ​​interest has changed and / or to what position it has changed.

[0067] In step 603, the optimization of the ultrasound imaging parameters is automatically adjusted according to the changes in the area of ​​interest. The processor can determine whether there is a new change area based on the changes in the area of ​​interest. If so, the ultrasound imaging parameters are adjusted to optimize the new area of ​​interest. Among them, the optimization method can be described in any of the embodiments above in this application. For example, more general parameters that can improve the quality of ultrasound images can be configured for the area of ​​interest, such as frame frequency, resolution, probe power, etc. The ultrasound imaging parameters can also be optimized for the area of ​​interest, such as focus, depth, time gain compensation and other parameters. Alternatively, image recognition can be performed on the area of ​​interest, and the anatomical features of the new area of ​​interest can be determined based on the results of image recognition, and special ultrasound imaging parameter optimization is provided for the anatomical features.

[0068] This configuration makes the ultrasound imaging system's optimization of ultrasound image quality more humane and more in line with the user's wishes. As the user's area of ​​interest changes, the optimization of ultrasound imaging parameters also changes instantly. On the one hand, it ensures that users can always observe high-quality images at any location; on the other hand, since the optimization of ultrasound imaging parameters is mainly aimed at the area of ​​interest rather than all ultrasound images, the resources of the ultrasound imaging system will not be excessively wasted and occupied, thereby ensuring that the high performance and stable operation of the system can be maintained sustainably.

[0069] In addition, the inventors further realized that during actual ultrasound imaging, for images of the same anatomical position, the user may need to move the probe and perform other operations to ensure that the best ultrasound data is obtained. Moving the ultrasound probe will inevitably cause changes in the field of view. Accordingly, the position of the ultrasound image displayed on the display will also change. In order to further improve the convenience of ultrasound imaging for users, improvements are also proposed in some embodiments of the present application.

[0070] Optionally, some other embodiments of the present application may further include step 605: during the ultrasound imaging process, the display position of the region of interest is kept fixed. That is, once the processor determines the region of interest, the position of the region of interest on the display is kept fixed. In this way, when the user needs to observe a certain region of interest but also needs to perform other operations such as moving the ultrasound probe, the region of interest will also be fixed at a specific position on the display, and the user can observe it without moving his eyes.

[0071] In some embodiments, step 605 can be used as a complete implementation method together with steps 601 and 603. For example, in a complete workflow, when the user's area of ​​interest is at a certain specific position, the ultrasound image of the area of ​​interest is fixed at a specific position on the display. Even if the user moves the probe, the ultrasound image of the area of ​​interest will not move, which can ensure the stability of the ultrasound image observed by the user. Reflecting on the user's intuitive feeling, it will not be affected by the change of the ultrasound image field of view, because the ultrasound image is fixed, and the user can feel the change in the quality of the ultrasound image brought by the moving probe, so that it is easier to obtain a satisfactory image. When the user ends the observation of the previous area of ​​interest and changes the position of the area of ​​interest, the ultrasound imaging parameters of the new area of ​​interest are first optimized and improved. Subsequently, the position of the new area of ​​interest will also be fixed at a specific position on the display, and the user can perform operations such as moving the probe to intuitively feel the further changes in the image quality. In other embodiments, step 605 can be implemented independently of steps 601 and 603, for example, it can be combined with any of the embodiments described above in this application.

[0072] Furthermore, some embodiments of the present application also provide an ultrasound imaging system. The ultrasound imaging system comprises:

[0073] A probe for acquiring ultrasound data about the tissue to be imaged;

[0074] A processor, wherein the processor is configured to perform the method described in any embodiment of the present application;

[0075] A display, wherein the display receives a signal from the processor and displays the signal;

[0076] The image acquisition unit is used to acquire the image of the user and send it to the processor.

[0077] The probe, display, and image acquisition unit in the ultrasound imaging system may be any of those described in the present application. For example, reference may be made to Figure 1 The description of the ultrasound imaging system shown may also be any other type in the art. The processor may also be Figure 1 The method executed by the processor of the controller circuit may be any of the methods described in any of the above embodiments of the present application, and unless explicitly excluded, the method steps of any of the above embodiments of the present application may be combined in any way.

[0078] Some embodiments of the present application also provide a non-transitory computer-readable medium, which stores a computer program, wherein the computer program has at least one code segment, and the at least one code segment can be executed by a machine so that the machine performs the steps of the method described in any of the above embodiments.

[0079] Accordingly, the present disclosure may be implemented in hardware, software, or a combination of hardware and software. The present disclosure may be implemented in a centralized manner in at least one computer system, or in a distributed manner in which different elements are distributed over several interconnected computer systems. Any type of computer system or other device suitable for implementing the methods described herein is suitable.

[0080] The various embodiments may also be embedded in a computer program product comprising all the features enabling the implementation of the methods described herein and which, when loaded into a computer system, is able to carry out these methods. A computer program in this context means any expression in any language, code or notation of a set of instructions intended to cause a system having information processing capabilities to perform certain functions, either directly or after either or both of the following: a) conversion into another language, code or notation; b) reproduction in a different material form.

[0081] The purpose of providing the above specific embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive, but the present invention is not limited to these specific embodiments. It should be understood by those skilled in the art that various modifications, equivalent substitutions and changes, etc., may also be made to the present invention, as long as these changes do not violate the spirit of the present invention, they should all be within the protection scope of the present invention.

Claims

1. An ultrasonic imaging method, comprising: generating and displaying an ultrasound image of the tissue to be imaged; Detecting the user's sight line and automatically determining the user's focus area on the ultrasound image; as well as The ultrasonic imaging parameters of the region of interest are optimized, and an optimized ultrasonic image for the region of interest is generated and displayed.

2. The method of claim 1, wherein: The detecting of the user's sight line comprises: Detecting the position on the ultrasound image toward which the user's sight line is directed, or detecting the position on the ultrasound image toward which the user's sight line is directed and the duration of time the user's sight line is directed at the position.

3. The method of claim 1, wherein: Optimizing the ultrasound imaging parameters of the region of interest includes: The ultrasound imaging parameters of at least a portion of the region of interest are optimized, and the imaging parameters of at least a portion of the ultrasound image outside the region of interest are not optimized.

4. The method of claim 1, wherein: The ultrasonic imaging parameters include at least one of ultrasonic data acquisition related parameters and ultrasonic image processing related parameters.

5. The method of claim 1, wherein: The automatic determination of the region of interest includes: Determining a focus point of the user on the ultrasound image based on detection of the user's line of sight; and An ultrasound image within a certain range around the point of interest is automatically determined as the region of interest.

6. The method of claim 1, wherein: The automatic determination of the region of interest includes: Determining a focus point of the user on the ultrasound image based on detection of the user's line of sight; performing image recognition on the ultrasound image of the point of interest to determine the corresponding anatomical features of the point of interest; and The entire ultrasound image corresponding to the anatomical feature is automatically determined as the region of interest.

7. The method of claim 6, wherein: The optimizing the ultrasound imaging parameters of the region of interest includes configuring the ultrasound imaging parameters according to the corresponding anatomical features.

8. The method of claim 7, wherein: The configuring the ultrasound imaging parameters for the corresponding anatomical features comprises at least one of the following methods: In response to the corresponding anatomical feature being smaller in size in the current ultrasound image, automatically enlarging and displaying the ultrasound image of the region of interest; In response to the ultrasound image being a volumetric image, an optimized viewing angle is automatically displayed for the corresponding anatomical feature.

9. The method of claim 1, wherein: Further including: Continuously detecting the user's sight line to automatically determine changes in the focus area; and According to the change of the region of interest, the optimization of the ultrasound imaging parameters is automatically adjusted.

10. The method of claim 1, wherein: Further including: The display position of the region of interest is kept fixed during the ultrasound imaging process.

11. An ultrasound imaging system, comprising: A probe for acquiring ultrasound data about the tissue to be imaged; A processor, the processor being configured to execute the method according to any one of claims 1 to 10; a display, the display receiving the signal from the processor and performing display; and The image acquisition unit is used to acquire the image of the user and send it to the processor.

12. The ultrasound imaging system of claim 11, wherein: The image acquisition unit is fixedly arranged on the display.

13. A non-transitory computer-readable medium storing a computer program, wherein the computer program has at least one code segment, and the at least one code segment can be executed by a machine to enable the machine to perform the steps of the method according to any one of claims 1 to 10.

Images