System and method for enabling untrained users to acquire ultrasound images of internal organs of the human body

Abstract

A system for enabling an untrained user to obtain ultrasound images of internal organs of a human body comprises a scanner, at least one inertial measurement unit (IMU) associated with the scanner, a processor including software, and a user interface comprising a display screen and means for accepting user instructions, the software being configured to perform at least one of: generating ultrasound images; analyzing the data; determining which images are of sufficient quality to be displayed on the display screen; discarding low-quality images; instructing the operator to hold the scanner housing in a predetermined manner; calculating the location and attitude of the scanner; determining whether the scanner is being held so that sufficient pressure is exerted on the skin to produce images of sufficient quality; and effectively providing instructions on how to properly move the scanner to obtain satisfactory images.

Description

[Technical Field]

[0001] The present invention originates from the field of medical devices. In particular, the present invention relates to a system and method for appropriately positioning and moving a handheld ultrasound probe using an inertial measuring device. [Background technology]

[0002] Knowing the location of medical sensors or devices relative to the patient's anatomical structure, and the speed at which the sensors or devices are moving or should be moving, is crucial for the functionality of advanced remote control, robotics, autonomous, self-feedback, or other automated medical procedures.

[0003] The speed at which the ultrasound probe (which will also be referred to herein as "scanner," "ultrasound head," or simply "probe" for simplicity) moves across the human body provides important information. For example, images taken slower or faster than a certain range may be deleted or, in some cases, may be subjected to special filtering and image processing techniques to enhance blurry images. Instructions may also be given to the operator regarding several aspects of the procedure, such as how to correct the scanning path, change orientation, or tilt the probe when stopping movement. When remotely controlling an ultrasound probe or sensor mounted on a gimbal positioned on the patient's body, knowing the two-dimensional or three-dimensional speed at which the ultrasound probe moves is also important for tracking the overall location, orientation, and speed of the gimbal and / or probe.

[0004] A very common procedure, among others, is the ultrasound scan, which almost all women receive during a prenatal visit to their primary care physician or clinic. Typically, in this scenario, an ultrasound technician (echocardiographer) or physician performs the scan. The operator, i.e., technician, midwife, primary care physician, or echocardiographer, knows from experience the best position and orientation in which the scanner head or probe must be positioned to image specific structures of the embryo, the correct amount of pressure on the abdomen necessary to maintain good adhesion of the scanner to the body, the angle of the probe relative to the abdomen, and the correct scanning speed that enables good imaging. Furthermore, the operator can view the images produced by the probe on the screen in real time and optimize or correct its position. In this specification, the terms “probe” or “ultrasound probe” refer to any useful probe, linear or convex phased array, HIFU, or other sensor.

[0005] In the context of this specification, the term “scanner” shall be understood to mean an element, housing, or device that must move over the surface of a patient’s body in order to acquire data from it, such as ultrasound images.

[0006] Many scans are performed simply to monitor signs of embryonic vitality, such as heart rate, movement, amniotic fluid volume, sounds, and respiration. These scans may also be performed by the patient at home or at locations other than a clinic, hospital, or medical facility, thus saving overburdened healthcare system time and resources, and potentially avoiding unnecessary visits to the emergency department or prenatal visits to a primary care physician's office, clinic, or hospital. However, performing ultrasound scans requires some skill from an ultrasound operator, which an untrained person does not possess. It is clearly desirable to provide a means for untrained individuals to perform “do-it-yourself” ultrasound scans that yield useful results. [Overview of the project] [Problems that the invention aims to solve]

[0007] Therefore, an object of the present invention is to provide a device and method that assists a patient in performing an ultrasound scan by monitoring the movement of the scanner head (ultrasound probe) and providing feedback that helps position the scanner head at a desired location.

[0008] Further objectives and advantages of the present invention will become apparent as the description progresses. [Means for solving the problem]

[0009] In a first embodiment, the present invention encompasses a system for enabling an untrained user to acquire ultrasound images of internal organs of the human body, comprising a scanner, at least one inertial measurement unit (IMU) associated with the scanner, a processor including software, and a user interface having a display screen and means for receiving user instructions, wherein the software is configured to perform at least one of the following: generating ultrasound images, analyzing data, determining which images are of sufficient quality to be displayed on the display screen, discarding low-quality images, instructing the operator to hold the scanner housing in a predetermined manner, calculating the location and orientation of the scanner, determining whether the scanner is held so that sufficient pressure is applied to the skin to produce images of sufficient quality, and effectively providing instructions on how to move the scanner correctly to obtain satisfactory images.

[0010] In one embodiment, the system comprises an electronic communication component selected from one or more of USB, Lightning, optical fiber, Wi-Fi, UWB, Bluetooth, and IR.

[0011] In a first embodiment, the present invention encompasses a system for acquiring ultrasound images of internal organs of the human body. The system comprises a scanner and at least one inertial measurement unit (IMU) associated with the scanner.

[0012] In some embodiments of the system, at least one IMU is one of the following: a) integrated with the scanner, b) connected to the scanner via a plug-in connection, or c) located within an element associated with the scanner and moving with the scanner during scanning.

[0013] Some embodiments of the system are configured to issue instructions to the system operator that allow scanning to be performed by individuals who have not received training in ultrasound scanning, including the patient themselves.

[0014] In some embodiments of the system, scanning is performed by untrained operators, and the scans are transmitted to a remote location for analysis by medical professionals.

[0015] Some embodiments of the system are configured to enable two-way communication between an operator and a remote individual or an unsupervised system, the unsupervised system comprising an automated image analysis circuit. Two-way communication can be selected from audio, visual, and video communication, or a combination thereof. In some embodiments of the system, when scanning is performed by an untrained operator, two-way video communication is enabled between the operator and a medical professional, allowing the operator and medical professional to see each other face-to-face while the operator is performing the scanning procedure, in order to assist the medical professional in interpreting the images and providing advice as needed. In some embodiments, the system is configured so that the system output is transmitted directly to a remote medical professional and / or an unsupervised system, either in real time or after the images have been acquired.

[0016] Some embodiments of the system are configured to overlay scanner images onto ultrasound scans to assist medical professionals in interpreting the images.

[0017] Embodiments of the system scanner include a housing that is ergonomically designed to be held by an operator and moved across the skin of a person or animal. Some embodiments of the housing include, or are associated with, at least a minimum number of components of the system that must be placed on the patient's body to obtain an ultrasound image. In some embodiments of the housing, the minimum number of components in or associated with the housing are i) an ultrasound probe head, ii) at least one IMU including a triaxial accelerometer and a triaxial gyroscope, iii) electronic components for wired or wireless communication with a remote terminal, and iv) a power supply.

[0018] When referring to "ultrasonic probe head," this term should be understood in its broadest sense to include, for example, single-element, 1D, 1.5D, 2D, 3D, and phased array probes, as well as any other configurations that can be used for the purposes of the present invention.

[0019] In some embodiments of the system, the housing comprises other components that can be arranged in many different configurations, and at least some of these other components may be arranged within the housing. In these embodiments, the other components of the system are: v) an analog front end (AFE) that transmits and receives ultrasonic signals by using electronic components, vi) a processor that includes software, vii) a user interface that includes a display screen and means for receiving user instructions, and viii) at least one memory device for storing data and images processed by the software in the processor. In these embodiments, the other components that are not arranged within the housing are arranged at a location that is near the patient but separated from the housing. In these embodiments, the other components that are not arranged within the housing communicate with components that are arranged within the housing or are associated with the housing.

[0020] In some embodiments of the system, the electronic components of the AFE include a transmitter, a receiver, an amplifier, and analog-to-digital (A / D) and digital-to-analog (D / A) converters.

[0021] In some embodiments of the system, the software is configured to operate the system, receive and process ultrasonic signals received from the AFE to generate ultrasonic images, and process inertial measurement signals received from the IMU.

[0022] In some embodiments of the system, the AFE, IMU, processor, memory device, and communication components can be provided as separate integrated circuits (ICs), or integrated into one or more ASICs that include at least some of these ICs.

[0023] Some embodiments of the system include additional components, which include ix) a remote terminal, x) at least one additional IMU, xi) at least one triaxial magnetometer, xii) at least one pressure sensor, and xiii) at least one of a speaker and a microphone for communicating with a remote healthcare provider.

[0024] In some embodiments of the system, all of the other components v) to viiii) are contained within a remote terminal connected to the scanner via a wired or wireless link. In other embodiments of the system, some of the other components v) to viiii) are contained within the scanner, and the remaining components are located within a remote terminal connected to the scanner via a wired or wireless link.

[0025] In some embodiments of the system, the remote terminal is a portable communication device. In some embodiments of the system, the portable communication device is a smartphone. In some embodiments of the system, the portable communication device comprises a display, an IMU, and a processor. In some embodiments of the system, the portable communication device is mated into a socket within the scanner housing. In some embodiments of the system, the portable communication device is an integral part of the housing. In some embodiments of the system, the portable communication device is not an integral part of the housing, but is mated into a socket within the housing before scanning is performed, moves with the housing during ultrasonic scanning, and is removed later for other use as desired. In some embodiments of the system, the portable communication device is connected to the housing via a cable or wireless connection, and only the housing moves.

[0026] Exemplary examples of suitable wired communication links include USB, Lightning, and fiber optic, but of course, any additional wired communication is possible. Exemplary examples of wireless communication links include, but are not limited to, Wi-Fi, UWB, Bluetooth, and IR.

[0027] The portable communication device can be any of many suitable devices, such as a mobile phone, tablet, or laptop. Furthermore, the housing or devices connected to the housing may be adapted to receive data generated by the housing, or may communicate with devices located in the cloud that are associated with the housing.

[0028] In some embodiments of the system, different combinations of one or more IMUs, processing devices and software, memory devices, power supplies, and AFE components are located either within the housing or within the smartphone. Some embodiments of the system include at least one IMU within the smartphone and at least one IMU within the housing.

[0029] In some embodiments of the system, the processor is configured to receive data collected by all sensors.

[0030] In some embodiments of the system, the software is configured to perform at least one of the following: generating ultrasound images, analyzing data, determining which images are of sufficient quality to be displayed on a screen, discarding low-quality images, instructing the operator to hold the scanner housing in a predetermined manner, calculating the scanner's location and orientation, determining whether the scanner is held so that sufficient pressure is applied to the skin to generate images of sufficient quality, and effectively providing instructions on how to move the scanner correctly to obtain satisfactory results.

[0031] The term "of satisfactory quality" refers to an image that allows for the estimation of anatomical structures or physiological conditions, either automatically or by expert observation, either as a whole or from its elements.

[0032] Some non-exclusive examples of things that can be inferred from images include fetal heart detection, fetal heart rate measurement, amniotic fluid measurement, fetal movement detection, measurement of blood flow direction and velocity in the fetal heart and arteries, detection of nodules in the thyroid gland, abnormal cells in the skin, abnormal prostatic structures, and detection of fluid in the lungs.

[0033] In some embodiments of the system, instructions to the operator, generated by software, are provided visually on a display screen or audibly through a speaker. In some embodiments of the system, instructions to the operator are provided visually on a display screen or audibly through a speaker by a trained medical professional located at a remote terminal.

[0034] In some embodiments of the system, the task of calculating navigation, including the scanner's location, orientation, and their time derivatives, is performed by an Inertial Navigation System (INS) comprising a set of three-axis gyroscopes and three-axis accelerometers and other sensors in an IMU, a processor, and software. The INS is configured to calculate navigation by taking initial conditions and calibration data, as well as outputs from the IMU and other sensors, the other sensors of which may be at least one of a three-axis magnetometer, a pressure sensor, and a camera.

[0035] Some embodiments of the system are configured to ensure images of sufficient value for diagnostic purposes by generating accurate scanning of ultrasound signals on the skin, using a combination of a pressure sensor and an IMU, and selecting only those images that meet optimal values ​​for scanner scanning speed and pressure on the skin.

[0036] In some implementations of the system, the INS provides the following types of data: a. Angle of direction, b. Scanner speed, and c. Location of the ultrasound probe head relative to the anatomical structure of the body.

[0037] In some embodiments of the system, the scanning speed is calculated from the angular velocity, assuming that the motion is perpendicular to the body's surface.

[0038] In some embodiments of the system, in the case of prenatal screening, the body is modeled as a sphere, and the radius of the sphere can be approximated by one or more of the following: the patient's BMI, the stage of pregnancy, or a visual estimate within the range of 20 cm to 70 cm for, for example, an obese patient.

[0039] In some embodiments of the system, the typical scanning distance is in the range of a few millimeters to several tens of centimeters. In some embodiments of the system, the scanning speed is in the range of 1 mm per second to several centimeters per second.

[0040] In some embodiments of the system, the IMU's triaxial gyroscope and triaxial accelerometer are calibrated by the manufacturer for offset, scale factor, cross axis sensitivity and initial orientation, and the MEMS IMU is calibrated by the user before each scan. In some embodiments of the system, if the scanning motion is slow and the operator maintains the scanner orientation relative to the body being scanned within a few degrees of a given starting orientation, a one-step calibration is required in which only the gyroscope offset is estimated. The one-step calibration process involves keeping the IMU stationary for several minutes and recording the sensor outputs, with the average output of the gyroscopes being interpreted as their offsets and the variance of each sensor being interpreted as its noise.

[0041] In some embodiments of the system, the operator performs a 7-phase calibration process in a coordinate system where the positive Z-axis points upward, the positive Y-axis points to the right, and the positive X-axis points forward. The seven phases of the calibration process are as follows: a. Phase 1: Hold the scanner stationary for T seconds. b. Phase 2: The scanner is rotated around the Y-axis within T seconds so that the rotation is completed and the scanner is stationary in the new orientation. c. Phase 3: Hold the scanner stationary for T seconds, then rotate it in the reverse direction. d. Phase 4: Rotate the scanner about the X-axis within T seconds. e. Phase 5: Hold the scanner stationary for T seconds, then rotate it in the reverse direction. f. Phase 6: The scanner is rotated about the Z-axis within T seconds. g. Phase 7: Hold the scanner still for T seconds, then rotate it in the reverse direction.

[0042] In some embodiments of the system, if the processor determines during scanning that insufficient pressure is being applied to the skin, an instruction to increase the pressure is issued to the operator either visually on the display screen, for example by displaying a downward arrow, and / or audibly through a speaker. In these embodiments, the processor can determine that insufficient pressure is being applied to the skin by at least one of the following: a. Analyze the image and determine that the photograph is flat. b. Measure the variance of image brightness across several regions of interest within the image and determine that the variance is less than a threshold, or c. Histogram thresholding, i.e., measuring the value of each pixel within the region of interest and determining the threshold at which an alert should be issued.

[0043] As used herein, the term “flat” means that insufficient pressure on the skin prevents the ultrasound from reaching the required depth, and as a result, the resulting image does not show the internal organs or fetus that the user is trying to image.

[0044] In some embodiments of the system, the processor includes software configured to determine whether the amount of aqueous gel placed between the ultrasound probe head and the skin is insufficient and to issue an alert to the operator either visually on a display screen and / or audibly from a speaker. In these embodiments, the software can determine whether the amount of aqueous gel placed between the ultrasound probe head and the skin is insufficient by determining whether there is a weakening of the signal returning to the probe or a weakening of the resulting ultrasound image.

[0045] In some embodiments, the system includes an IMU-independent component adapted to alert the user if the coupling between the device and the body is insufficient. In other embodiments, the system includes an IMU-independent component adapted to alert the user if the scanning speed is too fast.

[0046] In some embodiments of the system, the system's processor and software are configured to issue the following set of instructions to guide the operator to perform a scan. a. Guide the operator through the procedure and instruct them to perform calibration procedures when necessary. b. Instruct the operator to measure the patient's blood pressure using a blood pressure monitor. c. Instruct the operator to perform other analyses suggested by one or more details of the image. d. Instruct the operator on how to position the patient for scanning. e. Instruct the operator to position the scanner at a location that will serve as the center of the patient's coordinate system. f. Instruct the operator to position the scanner so that the screen faces the patient. g. Providing the operator with instructions including the direction in which the scanner should be moved over the patient's body surface, the distance to be moved in each direction, the speed at which the scanner should be moved, and the amount of force to be applied to compress the scanner against the body. h. Advising the operator that the session has ended when sufficient images of sufficient quality have been collected, and i. If this is not done automatically, advise the operator to transfer the images to a medical professional for interpretation.

[0047] In a second embodiment, the present invention encompasses a method for enabling an operator who has not been trained in ultrasound scanning to acquire and process ultrasound images of internal organs of the human body. The method includes: a. To provide a system comprising a scanner and at least one inertial measuring unit (IMU). The scanner is a component of the system moved by an operator over the surface of a patient's body to acquire an ultrasound image, at least one IMU is located within the scanner, and the system is configured to issue instructions to the operator of the system that enable scanning to be performed. b. Follow the instructions issued by the system.

[0048] In one embodiment of the method of the second aspect, the system is the system of the first aspect of the present invention.

[0049] In one embodiment of the method of the second aspect, the instruction issued by the system is an instruction issued by the processor and software of the system of the first aspect of the present invention.

[0050] In a third embodiment, the present invention encompasses a method for acquiring ultrasound images of internal organs of the human body. The method includes a scanner and at least one inertial measuring unit (IMU) associated with the scanner, as well as providing instructions for an untrained user to operate the scanner.

[0051] Some embodiments of the third aspect of the method include issuing instructions to the system operator that allow the scan to be performed by a person who has not been trained in ultrasound scanning, including the patient themselves. Some embodiments of the third aspect of the method include transmitting the acquired ultrasound images to a remote location for analysis by a medical professional. Some embodiments of the third aspect of the method include providing a circuit adapted to enable two-way communication between the user and a remote individual or an unsupervised system. In some embodiments of the third aspect of the method, the unsupervised system comprises an automated image analysis circuit, and the output of the automated analysis is provided to the user and / or a medical professional. In some embodiments of the third aspect of the method, the two-way communication is selected from audio, visual, and video communication, as well as combinations thereof.

[0052] In some embodiments of the third aspect of the method, scanning is performed by an untrained operator, and the system enables two-way video communication between the operator and a medical professional. In some embodiments of the third aspect of the method, the output of the system is transmitted directly to a remote medical and / or unsupervised system professional in real time or after the image has been acquired.

[0053] In some embodiments of the third aspect of the method, the system allows scanner images to be superimposed on ultrasound scans to assist medical professionals in interpreting the images.

[0054] In some embodiments, the method involves performing a calibration process on at least one axis in a coordinate system in which the positive Z-axis points upward, the positive Y-axis points to the right, and the positive X-axis points forward, the calibration process being as follows: a. Phase 1: Keep the scanner stationary for T seconds, then perform one or more of the following for each axis. b. Rotate the scanner so that the rotation is completed within T seconds around the selected axis and the scanner is stationary in the new orientation. c. Hold the scanner stationary for T seconds, then rotate it in the reverse direction.

[0055] In some embodiments of the third aspect of the method, the method includes performing a calibration process consisting of seven phases in a coordinate system in which the positive Z-axis points upward, the positive Y-axis points to the right, and the positive X-axis points forward, the seven phases being as follows: a. Phase 1: Hold the scanner stationary for T seconds. b. Phase 2: Rotate the scanner around the Y-axis so that the rotation is completed within T seconds and the scanner is stationary in the new orientation. c. Phase 3: Hold the scanner stationary for T seconds, then rotate it in the reverse direction. d. Phase 4: Rotate the scanner about the X-axis within T seconds. e. Phase 5: Hold the scanner stationary for T seconds, then rotate it in the reverse direction. f. Phase 6: The scanner is rotated about the Z-axis within T seconds. g. Phase 7: Hold the scanner still for T seconds, then rotate it in the reverse direction.

[0056] In some embodiments of the third aspect of the method, if the processor determines during scanning that insufficient pressure is being applied to the skin, an instruction to increase the pressure is issued to the operator either visually on the display screen, for example by displaying a downward arrow, and / or audibly from a speaker. In these embodiments, determining whether insufficient pressure is being applied to the skin can be done by at least one of the following: a. Analyze the image and determine that the photograph is flat. b. Measure the variance of image brightness across several regions of interest within the image and determine that the variance is less than a threshold, or c. Histogram thresholding, i.e., measuring the value of each pixel within the region of interest and determining the threshold at which an alert should be issued.

[0057] Some embodiments of the third aspect of the method include determining, through software analysis, whether the amount of aqueous gel placed between the ultrasound probe head and the skin is insufficient, and, if insufficient gel is found, issuing an alert to the operator either visually on a display screen and / or audibly from a speaker. In some embodiments of the third aspect of the method, the software can determine whether the amount of aqueous gel placed between the ultrasound probe head and the skin is insufficient by determining whether there is a weakening of the signal returning to the probe or a weakening of the resulting ultrasound image.

[0058] Some embodiments of the third aspect of the method include guiding the operator to perform a scan by issuing the following set of instructions. a. Guiding the operator through the procedure, thereby instructing the operator to perform the calibration procedure when necessary. b. Instruct the operator to measure the patient's blood pressure using a blood pressure monitor. c. Instruct the operator to perform other analyses suggested by one or more details of the image. d. Instruct the operator on how to position the patient for scanning. e. Instruct the operator to position the scanner at a location that will serve as the center of the patient's coordinate system. f. Instruct the operator to position the scanner so that the screen faces the patient. g. Providing the operator with instructions including the direction in which the scanner should be moved over the patient's body surface, the distance to be moved in each direction, the speed at which the scanner should be moved, and the amount of force to be applied to compress the scanner against the body. h. Advising the operator that the session has ended when sufficient images of sufficient quality have been collected, and i. If this is not done automatically, advise the operator to transfer the images to a medical professional for interpretation.

[0059] All of the above and other features and advantages of the present invention will be further understood through the following exemplary and non-limiting description of embodiments of the invention with reference to the accompanying drawings. [Brief explanation of the drawing]

[0060] [Figure 1] This figure shows four columns, each containing a plot of data related to the configuration process. [Figure 2] This figure shows the results of estimating the scanner orientation by applying an extended Kalman filter to calibrated gyroscope and accelerometer data. [Figure 3] This figure shows a repeated test similar to the one shown in Figure 2, but the measured values ​​are being supplied to EKF without calibration. [Figure 4] This figure shows the angular velocity during scanning and the tangential velocity derived from that angular velocity. [Figure 5] This diagram schematically shows an embodiment in which a smartphone equipped with the components of this system is fitted into a socket within the housing of the scanner of this system. [Figure 6] Figure 5 schematically shows a typical scene on the smartphone screen during scanning, according to an embodiment of the system shown. [Figure 7A] This is a screenshot illustrating the effects on images caused by insufficient coupling between the ultrasound probe head and the patient's body. [Figure 7B] This is a screenshot illustrating the effects on images caused by insufficient coupling between the ultrasound probe head and the patient's body. [Figure 7C]This is a screenshot illustrating the effects on images caused by insufficient coupling between the ultrasound probe head and the patient's body. [Figure 8] This is a screenshot showing the results of blood pressure measurements superimposed on a scan. [Figure 9] This diagram shows the movement of the scanner relative to the patient's body. [Figure 10] This is a flowchart of the combined alert process. [Modes for carrying out the invention]

[0061] Herein, the present invention is described as a system and method enabling patients to perform ultrasound scans themselves. Detailed examples in obstetrics and gynecology are given, but those skilled in the art can easily adapt these examples to other conditions and other organs, such as the cardiovascular system, lungs, kidneys, thyroid gland, liver, prostate, bladder, and other sensors. Moreover, although envisioned as a system for self-use in a home environment, its portability also allows for effective use by those who are not adequately trained in ultrasound scanning, such as by family members, in an ambulance, or by untrained military personnel on the battlefield. Needless to say, trained individuals can also benefit from using the present invention as a first approximation before other more sophisticated equipment becomes available to them.

[0062] Other sensors referenced above may include any type of sensor that generates data useful for improvement and / or adds relevant information to what is obtained through ultrasound imaging. For example, blood pressure (its importance in the context of the present invention will be discussed further) can be transmitted to the device of the present invention, in which blood pressure can be combined with or superimposed on other information, or used to alert the user and / or healthcare professionals of any potential problems. Another example is a proximity sensor that can be used to alert the user whether sufficient pressure is being applied to the body by the housing, which may result in faulty readings.

[0063] An additional example of a sensor useful in the context of the present invention is an image acquisition element that can be used independently of the IMU component to alert the user about coupling problems (e.g., due to insufficient pressure of the device against the body or insufficient gel) or when the user scans too fast and is unable to produce a good quality image. The above-mentioned situations and other situations requiring an alert to the user are detected via image processing that can be performed locally within the housing or remotely by a connected device.

[0064] The scan can be performed by the patient themselves and then transmitted to a remote location for analysis by a medical professional or an unsupervised system equipped with an automated image analysis circuit. Some embodiments of this system are configured to enable two-way video communication, i.e., telemedicine, which allows the patient and the ultrasound technician to see each other face-to-face while the patient is performing the scanning procedure.

[0065] The present invention also encompasses a system for acquiring and processing ultrasound images of internal organs of the human body. The system consists of many components arranged and configured in many different configurations, the examples of which are described herein. The components of the system that are essential to all configurations are referred herein to as “scanners,” and the scanners include components of the system that are moved by an operator over the surface of a patient’s body to acquire ultrasound images. Figure 9 shows possible forms of scanner movement relative to a patient’s body. The scanners comprise a housing that is held by an operator and is ergonomically designed to be moved across the skin of a person or animal. The housing comprises at least the minimum number of components of the system that must be placed on the patient’s body to acquire ultrasound images. These elements may be integrated with the housing or associated with the housing. In the context of this specification, the term “associated with” shall be interpreted as meaning that the element or component being referred to must not necessarily be integrated with the housing, but must work usefully with the housing. For example, when an accelerometer is discussed, the accelerometer must move with the housing, and when a communication component is discussed, the communication component must communicate with any other components located within the housing from which the communication component must exchange data or from which the communication component must receive data. These components include i) an ultrasonic probe head, i.e., an array of ultrasonic elements; ii) electronic components for wired or wireless communication with a remote terminal; and iii) a power source, such as a battery when the system is wireless or a power supply in the case of a wired system; and in most embodiments, iv) an inertial measuring unit (IMU) comprising inertial sensors, i.e., a triaxial accelerometer and a triaxial gyroscope, and optionally other sensors such as a triaxial magnetometer and a pressure sensor.

[0066] However, in some embodiments of the present invention, the inertial sensor is not integrated with the housing. Instead, an inertial sensor from a smartphone or similar portable device (described later) can be used, or an add-on inertial sensor can be connected to the housing before use. In another embodiment of the present invention, the housing can be a “docking housing,” i.e., a housing that includes only the components essential for connecting functional components such as various types of sensors to the housing, and the sensors can be connected to the docking housing as needed. This embodiment makes it possible to select the type of sensor appropriate for a given application, which can be added to the housing as a “plug-and-play” component.

[0067] Other typical components of the system include, in particular, a transmitter (pulsar), a receiver, an amplifier, and an analog front-end (AFE) that transmits and receives ultrasonic signals by electronic components including analog-to-digital (A / D) and digital-to-analog (D / A) converters; a processor including software configured to operate the system, receive and process ultrasonic signals received from the AFE to generate ultrasonic images, and receive and process inertial measurement signals received from the IMU; a user interface comprising a display screen and means for accepting user instructions, such as a keyboard or touchscreen; and a viii) one or more memory devices for storing data and images processed by the software in the processor. In different embodiments, some or all of these components may be located within the scanner housing or in a location near the patient but separated from the housing. As will be readily understood by those skilled in the art, there are many options for arranging and configuring these components.

[0068] The electronic components, namely AFE, IMU, processor, memory device, and communication components, can be provided as separate integrated circuits (ICs) or integrated into another ASIC that includes all or some of those ICs.

[0069] Optional components of the system include: ix) a remote terminal, such as a smartphone, tablet, PC, or similar communication / computing device, located near the operator or away from the operator, such as in a clinic or a doctor's office; x) one or more additional IMUs; x) at least one triaxial magnetometer; xi) at least one pressure sensor; and xi) a speaker and microphone for communicating with remote healthcare providers.

[0070] In some embodiments of the system, all components v) to viiii) are contained within the scanner housing (or on top of it, in the case of a display).

[0071] In some embodiments of the system, all components v) to viiii) are contained within a remote terminal connected to the scanner via a wired or wireless link, and the wireless link can be formed using any known technology, such as cellular, Wi-Fi, or Bluetooth.

[0072] In some embodiments of the system, some of components v) to viiii), such as some or all of the components of the AFE, are located within the scanner, while the remaining components are located within a remote terminal connected to the scanner via a wired or wireless link.

[0073] Figure 5 schematically shows an embodiment in which the display 10, IMU 12, and processor 14 are contained within a smartphone 16 that mates into a socket 18 within a housing 20 containing other components of the scanner. The smartphone 16 is not necessarily an integral part of the housing 20 and may be mated into the socket 18 before performing scanning, moved as an integral part of the housing 20 during ultrasonic scanning, and later removed for other uses. The smartphone 16 is electrically connected to the housing 20 by a connector 22 in the socket 18 that mates into a standard port on the smartphone 16. Figure 5 shows the ultrasonic probe head 24 at the bottom of the housing 20. As used herein, the term “smartphone” refers to any portable communication device that can create a mounting seat for itself within a housing such as the housing 20 in Figure 5, and is not intended to limit the present invention to any particular type of communication device, existing or to be developed. The smartphone was chosen in this example solely to illustrate the present invention because it is a widely available and popular device accessible to most people.

[0074] In another embodiment, the smartphone is connected to the housing via a cable or wireless connection, and only the housing or the probe itself is moved; that is, the smartphone does not necessarily need to move in conjunction with the ultrasonic probe.

[0075] In other embodiments, different combinations of one or more IMUs, processing devices and software, memory devices, power supplies, and AFE components are arranged either within the housing or within the smartphone.

[0076] On the one hand, IMUs are very noisy, and on the other hand, they are relatively inexpensive. Therefore, in some embodiments, it is advantageous to use some of them in one scanner, such as one IMU in a smartphone, and another IMU in a housing, or two or more IMUs in a housing. This increases the accuracy of positioning and motion measurements and improves the signal-to-noise (S / N) ratio of the received ultrasonic signal.

[0077] The processor is configured to receive data collected by all sensors and includes software configured to, among other things, generate ultrasound images, analyze the data, determine in some embodiments which images are of satisfactory quality to be displayed on a display screen, calculate the scanner's location and orientation, discard low-quality images, instruct the operator to hold the scanner housing in a predetermined manner, such as so that the display screen (or a designated symbol on the housing surface in embodiments where the display is remotely located) always faces the operator, determine whether sufficient pressure is being applied to the skin to generate a satisfactory quality image, and effectively provide instructions on how to move the scanner correctly by intuitive graphic cues presented on the display screen. In other embodiments, instructions to the operator are given visually or audibly on the display screen and speaker or by a trained medical professional located at a remote terminal.

[0078] Figure 6 schematically shows a typical scene on the screen of a smartphone 16 during scanning according to an embodiment of the system shown in Figure 5. In this exemplary embodiment, a blank area 26 is reserved on the screen for instructions from the system to the user. Typical instructions include, for example, the following: - The screen is not facing you - Please keep the screen perpendicular to your body. - The image is blurry - apply more pressure or add more gel. - The movement is too fast - please slow it down. - Move the housing to the right.

[0079] The task of calculating the scanner's location, orientation, and their time derivatives is performed by an inertial navigation system (INS). The INS consists of an IMU, which is a set of sensors including a triaxial accelerometer and a triaxial gyroscope, as well as other sensors such as a triaxial magnetometer and pressure sensor; a processor; and software configured to take initial conditions and configuration data, as well as outputs from the IMU and other sensors, and calculate navigation.

[0080] To improve accuracy, other sensors can be used in addition to the IMU, magnetometer, and pressure sensor. For example, a mobile phone has a front camera facing the user and a rear camera facing an object in the room. In this embodiment, the smartphone is fitted into a socket within a housing that also contains the other components of the scanner, and at the start of scanning, the rear camera is facing a specific object in the room. During scanning, the rear camera moves with the housing, and its movement relative to the object in the image can be tracked using the optical flow method, thereby providing the navigation algorithm with additional information that can be used to correct errors.

[0081] In embodiments of the present invention, the system can be configured to generate accurate scanning of ultrasonic signals on the skin, use a combination of a pressure sensor and an IMU, and select only those images that satisfy optimal values ​​for scanner scanning speed and pressure on the skin, thereby ensuring images of sufficient value for diagnostic purposes.

[0082] These sensors in an inertial measurement unit (IMU) or inertial navigation system (INS) can be implemented using a single-chip ASIC that includes all or some of them, or as separate chips, with each sensor implemented separately or as a combination of multiple sensors.

[0083] The IMU provides several types of data, including the following: 1. The orientation angle used for the following: a) Provide the user with instructions on how to hold the scanner to obtain the best image. b) To facilitate image interpretation, provide the physician or other specialist with the continuous orientation of the probe at the time the scan was performed. This information can be presented as an overlay on the ultrasound image. 2. The speed of the scanner used for the following purposes. a) Provide the user with instructions on how to move the scanner to obtain the best image. This information can be given to a physician located remotely, so that the physician can know how the scan is being performed through all the alerts received by the operator. b) Filtering out images that are unlikely to contain useful information. For example, the criteria for deleting images may be a speed greater than 10 cm / sec, or 1 cm / sec in situations where slow scanning is necessary to detect a specific phenomenon, such as self-scanning of the inferior vena cava (IVC) in patients with congestive heart failure (CHF). 3. Location of the ultrasound probe head relative to the anatomical structures of the body, used for the following purposes. a) Provide the user with instructions on how to scan the entire region of interest to fully cover the organ of interest. b) To facilitate image interpretation, provide the physician or other specialist with the sequential orientation of the scanner at the time the scan was performed.

[0084] IMUs, like any other device, are not perfect. IMU errors, under integration, form drift, i.e., errors that increase over time, and therefore, errors in the calculated location and orientation propagate rapidly over time. One example best illustrates this problem. Suppose that, due to measurement noise and other imperfections, the orientation of the device can be determined with an error of 1 milliradian. This error is considered very small, for example, given the quality of the IMU in a typical smartphone. Due to this error, the processor misinterprets the accelerometer reading and projects gravity at approximately 1 cm / sec. 2 This is interpreted as horizontal acceleration. As a result of this small acceleration error, a location error of 18 meters over one minute is produced, which is clearly well beyond the acceptable margin of error. Therefore, the processor must have some additional information and must assume some limitations in order to provide meaningful navigation.

[0085] The IMU installed inside a smartphone is based on Micro Electro-Mechanical System (MEMS) technology. MEMS technology provides tiny, efficient, and affordable sensors, but it has an inherent imperfection that leads to measurement errors. These errors can be divided into bias and noise. Formally, the only difference is that bias changes slowly, while noise changes rapidly. However, to illustrate this problem over the period associated with ultrasonic scanning, we can consider the bias to be constant and the noise to be absolutely random.

[0086] Therefore, due to bias, the IMU of a stationary device will still produce measurements that suggest the device is rotating and accelerating. Calibration procedures must be presented to calibrate the IMU and determine the bias. However, due to noise, calibration is never perfect, and some residual bias always remains. Furthermore, although the noise is random, it will only become zero after an infinite number of measurements. In practice, the expected value of the noise is the square root of the number of measurements multiplied by the standard deviation of the noise.

[0087] As described above, all IMUs installed in smartphones are MEMS-based and subject to strict limitations in cost, size, and energy consumption, and are therefore very similar to one another. Their noise and bias indices are, in principle, the same.

[0088] As a result of bias and noise, and given the quality of the MEMS IMU, the navigation process must integrate more measurements and utilize several prior assumptions to mitigate IMU errors. When scanning with a scanner, the distance moved is small and the scanning speed is generally low, and often, as a result, the noise generated within the IMU becomes larger than the signal. Typical scanning distances range from a few millimeters to tens of centimeters, and typical speeds range from 1 mm / sec to several centimeters per second. Therefore, the success of navigation depends on optimal calibration enabled by the system, mission, and user, as well as the integration of other available cues.

[0089] Some bias errors are calibrated at the manufacturing level. However, some biases change over time and must be calibrated before use. For the scanners described herein, the calibration process is limited to simple steps that can be easily performed by the user. A possible prior assumption is that the user cooperates by holding the scanner on a horizontal table facing the display.

[0090] When the scanner is positioned on a horizontal plane, the acceleration axis should be equal to 9.81 downwards; therefore, if a value different from 9.81 is measured, the processor can calibrate this offset and add it to each measurement. If the user requests that the IMU be calibrated, after the system has been activated and before starting a scanning session, the user will be instructed on how to calibrate the gyroscope and accelerometer, either by software within the processor or by a remotely located physician. Since the calibration value changes daily and each time the IMU is turned on, the IMU, especially IMUs created by MEMS technology, must be calibrated before each use.

[0091] Here, a calibration procedure consisting of seven phases is described. This procedure is just one of many that can be used with the scanner and is intended only to illustrate the principles involved. The inventors have used other calibration procedures consisting of fewer than seven phases, and other procedures consisting of different phase sequences or more or fewer than seven phases can be devised and used, for example, and the actual choice of calibration method is not essential as long as it yields the desired calibration result. In many situations, particularly when only slow motion is allowed and the user keeps the screen orienting themselves within a few degrees, a one-step calibration in which only the gyroscope offset is estimated yields excellent results. In this protocol, the IMU is held stationary for some time and the sensor outputs are recorded. The average output of the gyroscopes is interpreted as their offset, and the variance of each sensor is interpreted as its noise. The Earth's rotation of approximately 15 degrees per hour is usually negligible compared to the gyroscope offset.

[0092] For this example, a coordinate system is selected. In this coordinate system, the positive Z-axis points upward, the positive Y-axis points to the right, and the positive X-axis points forward. The letter T is used for the calibration duration, which can be, for example, 1, 3, 5, or 10 seconds or more, depending on the type of IMU. The value of T is a compromise between accuracy and user patience. This procedure has the following seven phases. Phase 1: Keep the scanner stationary for T seconds. Phase 2: Rotate the scanner around the Y-axis so that the rotation is completed within T seconds and the scanner is stationary in the new orientation. Phase 3: Hold the scanner stationary for T seconds, then rotate it in the reverse direction. Phase 4: Rotate the scanner around the X-axis within T seconds. Phase 5: Hold the scanner still for T seconds, then rotate it in the reverse direction. Phase 6: Rotate the scanner around the Z-axis within T seconds. Phase 7: Hold the scanner still for T seconds, then rotate it in the reverse direction.

[0093] During these seven phases, data from three accelerometers and three gyroscopes are collected by the electronic device and transferred to the processor. An example of gyroscope data is shown in Figure 1.

[0094] Figure 1 shows four columns, each containing plots of data related to the calibration process. In each column, the three rows refer to the three gyroscopes x, y, and z. In each plot in Figure 1, the horizontal axis is the measurement time, and the vertical axis is the measurement taken from the gyroscope or the error of this measurement. The vertical lines mark the boundaries between the seven phases, which are labeled as follows: S0 (1st), RY (2nd), SY (3rd), RX (4th), SX (5th), RZ (6th), and SZ (7th). Either the first letter S or R refers to either a "stationary" or "rotating" state. Either the second letter X, Y, or Z refers to the axis around which rotation occurs, or the axis around which rotation occurred before the stationary state.

[0095] Referring to Figure 1, we can see how the data is interpreted. The leftmost column contains the data collected from the gyroscope. In the first phase S0, for a time of 0-5 seconds, the device is stationary, and the gyroscope outputs its offset and any constant rotation, such as the Earth's rotation. In the second phase RY, for a time of 5-10 seconds, the first 2.5 seconds show a 180-degree rotation around the y-axis. Therefore, the y-gyroscope shows a large signal. The same applies to the other phases.

[0096] The next column, the second from the left, shows the measurement error. Note that in this example, the error is apparent because all gyroscopes detect either zero rotation or a known angular velocity of 180 degrees over a period of 2.5 seconds, i.e., approximately 1.26 rad / sec. Three characteristics of the signal can be seen in this column: a better-than-expected offset, the presence of a signal at the point of rotation on an axis other than the rotation axis, and a discrepancy between the rotation gyroscope output and the prediction. The latter two phenomena result from cross-axis measurement and scale factor errors.

[0097] In this example, the first phase S0, from 0 to 10 seconds, is stationary and therefore should result in zero, apart from the slight contribution of the Earth's rotation. However, in this example, the measurement is 10 seconds of the Earth's rotation. -4 The offset is approximately [0.12, -0.18, 0.02] rad / sec, including values ​​less than rad / sec. In the third phase, SY, data is acquired from the same sensor after it has been rotated 180 degrees around the y-axis, and in this case, the contributions of the Earth's rotation to x and z are reversed. Therefore, by averaging the data in S0 and SY, estimates of the x-gyro and y-gyro offsets are given. A similar protocol can be applied to other axes using other rotations.

[0098] The next column, the third from the left, shows the same data as the previous two columns after the calculated offset has been removed. Therefore, here the stationary state represents a value of 0 rad / sec with some noise added.

[0099] By examining the third column, we can calculate the cross-axis effect. For example, the output of the x-gyro during the RY phase should be zero, and is actually about 0.013 rad / sec. The ratio between the average output of the x-gyro, i.e., about 0.013 rad / sec, and the average output of the y-gyro, i.e., 0.126 rad / sec, is about 0.01, which generates a cross-axis effect between the y and x axes. Similarly, we can elucidate the relationships of all nine possible cross axes.

[0100] By comparing the error with the predicted result, the scale factor can also be calculated from the data in this column. For example, an error of 0.125 rad / sec is seen in the second row of phase RY. This error is approximately 0.1 of the signal, and therefore the scale factor is 1.1.

[0101] The scale coefficient and cross axis can be combined to form a matrix. By multiplying the original result by the inverse of this matrix and subtracting the original data, the result in the last column, which contains only noise, is generated. This noise is conveniently used to estimate the detector noise required by the extended Kalman filter.

[0102] It should be noted that the time-constant angular velocity shown here is merely for the purpose of clarifying this example. The same result can be obtained by replacing all calculations with their averages over the duration of the phase.

[0103] The algorithm used by the processor's software to calibrate the gyroscope offset is as follows: 1. O_s0 = the average of the data collected in Phase 1 for each of the three gyroscopes. 2. O_sy = the average of the data collected in Phase 3 for each of the three gyroscopes. 3. O_sz = the average of the data collected in Phase 7 for each of the three gyroscopes. 4. Calculate the following:

number

[0104] The algorithm used by the processor's software to calibrate the gyroscope's scale factor is as follows: 5. O_ry = the average of the data collected in Phase 2 for each of the three gyroscopes. 6. O_rx = the average of the data collected in Phase 4 for each of the three gyroscopes. 7. O_rz = the average of the data collected in phase 6 for each of the three gyroscopes. 8. Calculate the following: [Number]

[0105] The algorithm used by the software in the processor to calibrate the cross-axis sensitivity of the gyroscope is based on matrix C. ω It is based on 9. Here, [Number]

[0106] The algorithm used by the software in the processor to calculate the three projections of gravity on the three accelerometers in the initial body coordinates is as follows. 1. A_s0 = The average of the data collected in phase 1 for each of the three accelerometers. 2. A_sy = The average of the data collected in phase 3 for each of the three accelerometers. 3. A_sz = The average of the data collected in phase 5 for each of the three accelerometers. 4. Calculate the following. a) x projection of gravity = A_ref x =(A_s0 x - A_sy x ) / 2 b) y projection of gravity = A_ref y =(A_s0 <​​​​​​​​​​​​​​​​​​​​​​ It consists of seven members, namely, three components of angular velocity in body-frame coordinates.

number

number

number

number

number

[0109] The measurement vector is as follows:

number

number

number

number

number

number

[0110] Similarly,

number

number

[0111] C a and B a and C ω and B ω This is calculated during the calibration process.

[0112] Implicitly, this filter is,

number

[0113] Figure 2 shows the results of estimating the scanner orientation by applying an extended Kalman filter to calibrated gyroscope and accelerometer data. The leftmost column shows the gyroscope output by dotted lines, the extended Kalman filter (EKF) estimate of rotation by dashed lines, and the true rotation by solid lines. Each row represents one axis, x, y, and z. The leftmost column relates to angular velocity as measured in body-fixed coordinates. For example, looking at the x-gyro at the top, the true rotation is zero. The sensor produces approximately -0.18 rad / sec resulting from the offset. Both the true rotation and the calibrated signal are close to zero. Note that for the y-axis, a true rotation of 0.1 rad / sec is applied. All measurements contain some noise of a magnitude of several milliradians per second. Noise is more easily seen in the z-gyro output, as the scale of the numbers shrinks to the noise level in the absence of a large offset or rotation. The rightmost column shows the four elements of the quaternion used in the EKF to estimate orientation. Here again, the solid and dashed lines are used for the actual and estimated quaternions, and they are very close to each other. The middle column shows the orientation in Euler angles, which is easier to interpret. Since an angular velocity of 0.1 rad / sec is applied, the y angle advances at this velocity. The solid and dashed lines are so close that they are indistinguishable. The dynamics of the error can be seen better in the x and z angles, where some error accumulates as the y angle approaches 90 degrees. Of course, 180 degrees and -180 degrees refer to the same angle and are not errors. The accumulation of error as the y rotation approaches 90 degrees is not accidental but arises from a numerical effect. The conversion of quaternions to Euler angles uses inverse trigonometric functions and is very sensitive around 90 degrees.

[0114] Figure 3 shows a repeating test similar to that shown in Figure 2, but with the measured values ​​supplied to the EKF without calibration. Errors in the signal, offsets on the y-axis, and cross-axis effects on other axes can be observed. These errors are converted into a large error in the y-angle and observable errors in the x and z angles.

[0115] Ultrasound scanning relies on holding the scanner so that some pressure is applied to the skin. As the pressure decreases, the scanner produces a flat image. The processor analyzes the image and, based on this, concludes that the photograph is flat, or, instead of using the entire photograph, uses similar criteria, such as measuring the variance of brightness in the image across a region of interest. If the brightness is below a threshold, the processor issues an instruction to the operator to increase the pressure. In one embodiment, this instruction may include, as an example, the appearance of a downward arrow on the display screen, accompanied by an audible instruction to increase the pressure on the skin.

[0116] To provide a smooth medium for the ultrasound beam to propagate from the probe to the body, aqueous gels are commonly used; otherwise, the beam is attenuated as it passes through the air. The resulting signal or image can be used to determine whether the coupling between the probe and the body is satisfactory. This can be determined, for example, by the weakening of the signal returning to the probe or the resulting ultrasound image. Figure 7A is a screenshot showing good coupling between the ultrasound probe head and the patient's body, while Figures 7B and 7C show examples of insufficient or partial coupling. This process can be performed within the mobile device's processor, or within the AFE's controller, within the device components including the ultrasound transducer, or within external software.

[0117] The scanning speed can be calculated from the angular velocity. The processor assumes motion perpendicular to the body surface. In the case of prenatal screening, the body can be modeled as a sphere, for example, with R0 = 20, 30, 40, or even 70 cm for obese patients. The radius can be better approximated based on the patient's BMI and stage of pregnancy. The velocity can be approximated as follows:

number

number

number

number

number

number

[0118] Figure 4 shows the angular velocity under which the scan is performed, and the spatial velocity derived from the angular velocity. The third column of the figure shows the three components of the velocity in radial coordinates along the abdomen of a pregnant patient. The X-axis refers to the radial motion from the center of the abdomen outward. This motion is assumed to be zero. The Y-axis refers to the motion across the abdomen from bottom to top, and the Z-axis refers to the motion from right to left. The other columns show the same information as the tree columns in Figure 2 and are shown for reference. The acceptable velocity range is a characteristic of the scanner and is typically a few centimeters per second. This slow motion generates a small radial acceleration of about 1 millimeter per square second, which means that the acceleration due to gravity can be used as a good approximation of the downward acceleration by EKF. Therefore, when the calculated velocity is not within the acceptable range, the scan is discarded and the patient is instructed to slow down further.

[0119] By combining speed and orientation, the scanner can ensure that the user is instructed to cover a predetermined angular range and do so within an acceptable speed range. In addition to the quality of the images produced by image processing, appropriate pressure on the skin is also maintained. In summary, this ensures a good examination.

[0120] In many cases, physicians or other trained medical professionals either directly observe the scan results or are given the scan for analysis, so it is important that they are given all the information necessary to understand the data provided. In the case of prenatal scans, blood pressure is measured at every prenatal visit. Therefore, at home, the patient's blood pressure should also be measured, and the measurement results should be added to the recording of the ultrasound scan. High blood pressure during pregnancy is an important clinical outcome and indicator of pre-eclampsia, and is crucial in determining how the remainder of pregnancy will be managed before delivery, the timing of delivery, the risk of complications, and long-term maternal morbidity. A low fetal heart rate and low maternal blood pressure may indicate a healthy fetus, while a low fetal heart rate and normal maternal blood pressure may indicate a diseased fetus, which also influences how the ultrasound technician associates the scan. Figure 8 is a screenshot showing one example of how the results of blood pressure measurements can be displayed to a physician or other trained medical professional, both as a written message and as an overlay on the scan.

[0121] The scanner is a "black box" as far as the scanner operator is concerned. All of the algorithms described above are useful only for the internal operation of the system, and its processor is programmed to utilize those algorithms to generate instructions for the patient to guide them through the process of acquiring ultrasound scans of satisfactory quality to provide useful information. The patient only needs to follow the visual or auditory instructions they receive from the system components or, in the case of telemedicine, from the ultrasound technician. It is also possible to show video instructions through animation.

[0122] Generally, a typical set of instructions issued by the system to guide the operator through the scanning process includes the following: a) If necessary, instruct the patient to perform a calibration procedure by guiding the patient through a procedure such as the single-step or seven-step calibration procedure described herein. b) Instruct the patient to measure their blood pressure using a blood pressure monitor. c) Instruct the patient to undergo additional tests. d) For example, instructing the patient on how to position themselves for the scan, such as horizontally on the back for a prenatal scan. e) Instructing the patient to position the scanner at a location that serves as the center of the patient's coordinate system, for example, above the umbilicus for prenatal screening, between the nipples for cardiac scanning, or three finger-widths from the right or left nipple for lung scanning. f) Instruct the patient to position the scanner so that the screen faces the patient. g) Providing the patient with instructions including the direction in which the scanner should be moved on the surface of the patient's body, the distance to be moved in each direction, the speed at which the scanner should be moved, and the amount of force to be applied to compress the scanner against the body. h) Inform the patient that the session is over when sufficient images of sufficient quality have been collected, and i) If this is not done automatically, advise the patient to transfer the images to a medical professional for interpretation.

[0123] In some embodiments of the present invention, the scanner output may be transmitted directly to a medical professional, such as the patient's attending physician, in real time or after the output has been acquired, and some or all of the instructions to the patient may be transmitted by the physician, particularly when a particular area of ​​anatomical structure needs to be studied more deeply than is normally possible from a general scan. As an aid to the physician, in some embodiments of the system, software within the processor is configured to overlay the scanner image onto the ultrasound scan. In other embodiments, the processor is configured to relay instructions transmitted to the operator during the scan, so that the physician can understand which instructions regarding the image have been presented and when they were presented.

[0124] "Example 1": Join alert The following illustrates a combined alert procedure according to one specific embodiment of the present invention. The procedure includes the following steps: a. Image acquisition - Construction of ultrasound images from echoes received from body organs to the transducer. b. Image preprocessing - At the start of the process, frames undergo image preprocessing to normalize the variance between frames from multiple different scans. c. Total Black Frame (TBF) Test - After image preprocessing, the algorithm performs a TBF test. In the TBF test, the percentage of pixels that are absolutely black within the entire current frame is tested to find frames that meet the TBF criteria. d. Joint condition classification - The join conditions for any side (left / right) of each frame are created by the decision tree classifier. e. Buffer Test - Each classification is stored in a buffer for the length of 16 decisions. If 80% of the decisions show insufficient binding, the user is instructed to improve skin contact or add more gel. f. Display alerts to the operator - While performing the scan, the user receives real-time feedback on the bonding conditions. If 80% of the frames have insufficient bonding, the user is instructed to improve skin contact or add more gel. g. Add the image to the recording - If a good match is detected, the frame will be recorded. h. Display an alert to the operator (TBF) - If no binding is identified, the system will instruct the user to hold the cradle more closely against the skin. i. Drop images from the recording - therefore, if the frame is not in the TBF case, the received image will be improved. j. Display images on the screen - All images are displayed on the screen (TBF, poorly coupled, and well coupled).

[0125] This process is shown in the form of a flowchart in Figure 10.

[0126] "Example 2": "Scanning too fast" alert The following steps address users who are moving the housing too quickly, preventing them from generating good quality scans.

[0127] The following two steps are performed to obtain the scanning speed value from the scanned image. a. To detect changes in the majority of the image, and b. Detecting optical flow to obtain velocity.

[0128] The first step aims to distinguish between embryonic movement and scanner movement. Embryonic movement is localized, and therefore, their movement does not change the majority of the image. In contrast, scanner movement changes the entire image at once. To estimate the change, the standard deviation over time is calculated over 6 frames. If a significant change is detected in more than 0.5% of the total scanned pixels, this indicates that movement has occurred.

[0129] To assess the overall changes within the image, the change in pixel intensity per second is evaluated across the image. The pixel standard deviation over time is used as a means of estimating the changes. For image I(x,y,n), "n" is the number of frames, and each frame is captured at time t(n). The following calculations are used to assess the changes.

number

[0130] This provides a measurement of the amount of change per frame. To evaluate changes over time, the values ​​are normalized by the average FPS.

number

[0131] In the next step, the number of pixels that have changed dramatically will be calculated.

number

[0132] Here, the sum of C is calculated to understand what percentage of the image has changed.

[0133]

number

[0134] For moving frames, use the Lucas-Kanade method with pyramids to perform optical flow V x , V y The result is calculated. The central corner of the image is used in the calculation using the Harris corner detector.

[0135] Optical flow gives the velocity per frame. To reach that velocity within a given time, the velocity should be normalized by FPS.

number

[0136] While embodiments of the present invention have been described using examples, it will be understood that the present invention may be carried out with many changes, modifications, and adaptations without exceeding the scope of the patent claims.

Claims

1. A system for enabling an untrained user to acquire ultrasound images of internal organs of the human body, comprising a scanner, at least one inertial measuring unit (IMU) associated with the scanner, a processor including software, and a user interface having a display screen and means for receiving user instructions, wherein the software is configured to provide instructions on how to move the scanner correctly across the skin of the human body to obtain images depicting anatomical structures or physiological conditions. The processor and software of the system provide instructions to the operator to perform a scan. a. Guiding the operator through the calibration procedure, and instructing the operator to perform the calibration procedure when necessary. b. Instructing the operator to measure the patient's blood pressure using a blood pressure monitor. c. Instructing the operator to perform other analyses suggested by one or more details of the image. d. Instructing the operator on how to position the patient for the scan, e. Instructing the operator to position the scanner at a location that serves as the center of the patient coordinate system. f. Instructing the operator to operate the scanner so that the screen faces the patient, and g. Providing the operator with instructions including the direction in which the scanner should be moved over the surface of the patient's body, the distance to be moved in each direction, the speed at which the scanner should be moved, and the amount of force to be applied to compress the scanner against the body. h. Advising the operator that the session has ended when a sufficient number of images of sufficient quality have been collected. A system configured to issue a set of instructions.

2. The system according to claim 1, wherein if the processor determines during scanning that sufficient pressure is not being applied to the skin, an instruction to increase the pressure is issued to the operator either visually on a display screen and / or audibly from a speaker.

3. The aforementioned processor, a. Analyze the image and determine that the photograph is flat, and b. Measure the variance of the brightness of the image across several regions of interest within the image and determine that the variance is less than a threshold, or c. Histogram thresholds, i.e., measuring the value of each pixel within the region of interest and determining the threshold at which an alert is required. The system according to claim 2, wherein it is determined that sufficient pressure is not being applied to the skin by at least one of the following.

4. The system according to claim 1, wherein the processor includes software configured to determine whether the amount of aqueous gel placed between the ultrasonic probe head of the scanner and the skin is insufficient, and to issue an alert to the operator either visually on the display screen and / or audibly from a speaker.

5. The system according to claim 4, wherein the software determines whether there is a weakening of the signal returning to the probe, or whether there is a weakening of the resulting ultrasound image, thereby determining whether the amount of aqueous gel placed between the ultrasound probe head and the skin is insufficient.

6. The system according to claim 1, comprising an electronic communication component selected from one or more of the following: a universal serial bus, optical fiber, wireless local area network, ultra-wideband radio, and infrared.

7. The system according to claim 1, comprising a component adapted to alert the user if the connection between the device and the patient's body is insufficient, wherein the component does not depend on data generated by the IMU.

8. The system according to claim 1, comprising a component adapted to alert the user if the scanning speed is too fast, wherein the component does not depend on data generated by the IMU.

9. A method for acquiring an ultrasound image of an internal organ of the human body, comprising providing the system described in claim 1 to an operator who has not been trained in ultrasound scanning, and operating the scanner according to a set of instructions to acquire an ultrasound image of an internal organ of the human body.

10. The method according to claim 9, comprising transmitting acquired ultrasound images to a remote location for analysis by a medical professional.

11. The method according to claim 9, comprising providing a circuit adapted to enable bidirectional communication between the user and a remote individual or system.

12. The method according to claim 11, wherein the system comprises an automatic image analysis circuit, and the output of the automatic analysis is provided to the user and / or a medical professional.

13. The method according to claim 11, wherein the two-way communication is selected from audio, visual, and video communication, and combinations thereof.

14. The method according to claim 9, wherein the scanning is performed by an operator and includes two-way video communication between the operator and a medical professional.

15. The method according to claim 14, comprising transmitting the output of the system directly to a remote medical professional and / or system in real time or after the image has been acquired.

16. The method according to claim 9, comprising overlaying the scanner image onto the ultrasound scan to assist a medical professional in interpreting the ultrasound scan.

17. The calibration process includes performing a calibration process on at least one axis in a coordinate system where the X, Y, and Z axes are orthogonal to each other, and the calibration process is as follows: a. Phase 1: The scanner is kept stationary for a certain time T seconds, and then, for each axis, one or more of the following are performed: b. Rotating the scanner around the selected axis within the time T seconds so that the rotation is completed and the scanner is stationary in the new orientation, and c. The method according to claim 9, wherein the scanner is held stationary for the time T seconds and then rotated in the reverse direction.

18. The method according to claim 9, wherein if the processor determines during scanning that sufficient pressure is not being applied to the skin, an instruction to increase the pressure is issued to the operator visually on a display screen and / or audibly from a speaker.

19. a. Analyze the image and determine that the photograph is flat. b. Measure the variance of image brightness across several regions of interest within the image and determine that the variance is less than a threshold, or c. Histogram thresholds, i.e., measuring the value of each pixel within the region of interest and determining the threshold at which an alert is required. The method according to claim 18, comprising determining whether sufficient pressure is being applied to the skin by at least one of the following.

20. The method according to claim 9, comprising determining, through software analysis, whether the amount of aqueous gel placed between the ultrasound probe head and the skin is insufficient, and, if insufficient gel is found, issuing an alert to the operator visually on a display screen and / or audibly from a speaker.

21. The method according to claim 20, wherein the software determines whether there is a weakening of the signal returning to the probe, or whether there is a weakening of the resulting ultrasound image, to determine whether the amount of aqueous gel placed between the ultrasound probe head and the skin is insufficient.

22. This involves guiding the operator to perform a scan, using the following set of instructions: j. Guiding the operator through the calibration procedure, and instructing the operator to perform the calibration procedure when necessary. k. Instruct the operator to measure the patient's blood pressure using a blood pressure monitor. l. Instruct the operator to perform other analyses suggested by one or more details of the image. m. Instructing the operator on how to position the patient for the purpose of performing the scan. n. Instructing the operator to position the scanner at a location that serves as the center of the patient coordinate system. o. Instructing the operator to position the scanner so that the screen faces the patient, and p. Providing the operator with instructions including the direction in which the scanner should be moved over the surface of the patient's body, the distance to be moved in each direction, the speed at which the scanner should be moved, and the amount of force to be applied to compress the scanner against the body. q. Advise the operator that the session has ended when sufficient images of sufficient quality have been collected. The method of claim 9, comprising guiding an operator to perform a scan by issuing an instruction.

23. The method according to claim 22, comprising advising the operator to transfer the images to a medical professional for interpretation.

Images