Ultrasound Qualification System

Abstract

To provide a system and a method for automatic supersonic wave qualification authorization.SOLUTION: There is provided a qualification authorization system for issuing supersonic wave examiner qualification authorization to a supersonic wave examination candidate. A supersonic wave system 101 includes a computing device 102, and a supersonic wave probe 103 connected to the computing device to generate supersonic wave data. The computing device generates a supersonic wave examination score as a part of automatic examination on the basis of the supersonic wave data, and transfers the supersonic wave examination candidate from automatic examination to manual examination by an examiner on the basis of the supersonic wave examination score.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The embodiments disclosed herein relate to ultrasound systems. More particularly, the embodiments disclosed herein relate to ultrasound qualification systems. [Background technology]

[0002] Generally, ultrasound systems generate ultrasound images by transmitting sound waves at frequencies above the audible spectrum into the body, receiving echo signals resulting from the sound waves reflecting off parts of the body, and converting the echo signals into electrical signals for image generation. Because they are non-invasive and can provide immediate imaging results, ultrasound systems are widely used in nursing homes. In most of these nursing homes, the demand for certified ultrasound operators far exceeds the number of available certified ultrasound operators.

[0003] This discrepancy in the demand and supply of certified ultrasound operators is due, in part, to the extensive training and review process of the credentialing systems used by care facilities to credential ultrasound operators. For example, candidates (e.g., students) training to become certified ultrasound operators often must perform ultrasound examinations on a variety of patients with various medical conditions and submit the ultrasound data (e.g., imaging results) for manual review and approval by reviewers at the care facility. This review process typically does not allow reviewers to provide real-time feedback to candidates during the ultrasound examination because reviewers are not able to review each candidate's ultrasound data during the ultrasound examination. This problem is exacerbated when multiple candidates are simultaneously generating ultrasound examination data for review, for example, based on different patients within the care facility. Thus, candidates may not receive feedback from the reviewer until much after the ultrasound examination, such as days or weeks. Thus, the ultrasound examination may not be fresh in the candidate's memory when the candidate receives the reviewer feedback, which may result in the candidate being unable to fully utilize the feedback and slowing down the candidate credentialing process.

[0004] Because traditional credentialing systems rely on human judgment, they are necessarily subjective and often inaccurate. This is because a reviewer may determine ultrasound data is acceptable when it is not, and vice versa. Furthermore, traditional credentialing systems are generally not standardized across reviewers (within a care facility and / or across different care facilities), so one reviewer may determine ultrasound data is unacceptable while another reviewer may determine the same ultrasound data is acceptable. Additionally, the reliance on reviewers (e.g., one or more senior clinicians) places a significant resource burden on the care facility in terms of the time it takes for the reviewer to review candidate submissions and generally oversee the credentialing program.

[0005] Furthermore, for ultrasound examinations that require emergency medical attention (e.g., determining free fluid in a patient), candidates may not be able to submit ultrasound data for review based on examinations of live patients. In these cases, the certification system may generate mock ultrasound images via training or a simulator system, and candidates may submit the mock ultrasound images to the certification system reviewer. However, reviewers have an inherent bias when grading the mock ultrasound images, since they are usually aware that mock ultrasound images are not true ultrasound images. Thus, reviewers may grade the mock ultrasound images as acceptable because they are mock images rather than true ultrasound images.

[0006] Thus, traditional ultrasound credentialing systems introduce delays into the credentialing process, burden care facilities, and can result in poorly trained but nevertheless credentialed ultrasound operators, and thus patients needing ultrasound examinations may receive less than the best care available. Summary of the Invention

[0007] Systems and methods for automated ultrasound credentialing are described. In some embodiments, a credentialing system for issuing sonographer credentialing to ultrasound candidates includes a computing device and an ultrasound probe connected to the computing device and configured to generate ultrasound data. The computing device is configured to generate an ultrasound score as part of the automated review based on the ultrasound data. The computing device is configured to transfer the ultrasound candidate from the automated review to a manual review by a reviewer based on the ultrasound score.

[0008] In some embodiments, a credentialing system for issuing a sonographer credential to a sonography candidate includes an ultrasound system configured to generate ultrasound images, the credentialing system includes a candidate credentialing application implemented at least in part in hardware of the credentialing system and configured to generate an image quality score for a first subset of the ultrasound images and communicate a second subset of the ultrasound images to an examiner computing device based on the image quality score.

[0009] In some embodiments, a ultrasound examination qualification system includes an ultrasound probe configured to generate ultrasound data. The ultrasound examination qualification system includes a computing device configured to generate ultrasound images based on the ultrasound data. The ultrasound examination qualification system includes a neural network implemented at least in part in hardware of the computing device to generate an image quality score based on the ultrasound images. The ultrasound examination qualification system includes a qualification device configured to issue a sonographer qualification based on the image quality score.

[0010] In some embodiments, a method implemented by a computing device includes receiving ultrasound data from an ultrasound probe connected to the computing device. The method includes generating, with the computing device, an image quality score based on the ultrasound data. The method includes communicating the ultrasound data to an examiner computing device for user qualification based on the image quality score.

[0011] In some embodiments, a method implemented by a computing device includes receiving ultrasound data from an ultrasound probe connected to the computing device. The method includes generating, with the computing device, an image quality score based on the ultrasound data. The method includes discontinuing communication of the ultrasound data to an assessor computing device for user qualification based on the image quality score.

[0012] In some embodiments, a method implemented by a computing device includes receiving ultrasound data from an ultrasound probe connected to the computing device. The method includes generating, with the computing device, an image quality score based on the ultrasound data. The method includes communicating the image quality score and the ultrasound data to a certification server for user certification.

[0013] In some embodiments, a method implemented by a computing device includes receiving ultrasound images. The method includes generating an image quality score for a first subset of the ultrasound images. The method includes communicating a second subset of the ultrasound images to an examiner computing device for user qualification based on the image quality score.

[0014] In some embodiments, a method implemented by a computing device includes receiving an ultrasound image. The method includes generating an image quality score for the ultrasound image using a neural network implemented at least in part in hardware of the computing device. The method includes issuing an ultrasound examination credential based on the image quality score.

[0015] Other systems, machines, and methods for ultrasound qualification are also described.

[0016] The accompanying drawings depict examples and are therefore exemplary embodiments and are not to be considered as limiting the scope. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 illustrates an environment for implementing a credentialing system according to some embodiments. [Diagram 2] FIG. 2 illustrates an ultrasonic probe having a sensor region for determining grip orientation in accordance with some embodiments. [Diagram 3] FIG. 3 illustrates a system for generating data related to candidate evaluation based on one or more ultrasound images and one or more secondary inputs in accordance with some embodiments. [Figure 4] FIG. 4 is a data flow diagram of a process performed by a credentialing system to issue sonographer certification to sonography candidates in accordance with some embodiments. [Diagram 5] FIG. 5 is a data flow diagram of a process performed by a credentialing system to issue sonographer certification to sonography candidates in accordance with some embodiments. [Figure 6] FIG. 6 is a data flow diagram of a process performed by an ultrasound examination qualification system according to some embodiments. [Figure 7] FIG. 7 is a data flow diagram of a method implemented by a computing device according to some embodiments. [Figure 8] FIG. 8 is a data flow diagram of a method implemented by a computing device according to some embodiments. [Figure 9] FIG. 9 is a data flow diagram of a method implemented by a computing device according to some embodiments. [Figure 10] FIG. 10 is a data flow diagram of a method implemented by a computing device according to some embodiments. [Figure 11] FIG. 11 is a data flow diagram of a method implemented by a computing device according to some embodiments. [Figure 12]FIG. 12 is a block diagram of an example computing device capable of performing one or more of the operations described herein, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Systems and methods for automated ultrasound credentialing are described. In some embodiments, a credentialing system for issuing sonographer credentialing to ultrasound candidates includes a computing device and an ultrasound probe connected to the computing device and configured to generate ultrasound data. The computing device is configured to generate an ultrasound score as part of the automated review based on the ultrasound data. The computing device is configured to transfer the ultrasound candidate from the automated review to a manual review by a reviewer based on the ultrasound score.

[0019] Traditional ultrasound credentialing systems can result in delays in the credentialing process, strain nursing home resources, and can lead to ultrasound operators being credentialed when they are not adequately trained, resulting in patients needing ultrasound examinations receiving less than the best care available.

[0020] The embodiments of the systems, devices, and methods for ultrasound credentialing disclosed herein constitute many advantages over conventional credentialing systems. The embodiments disclosed herein eliminate bias and improve the speed of credentialing compared to conventional ultrasound credentialing systems. The embodiments disclosed herein facilitate real-time feedback to the candidate during the ultrasound examination, allowing the candidate to immediately incorporate feedback without delay into the ultrasound examination while the examination is fresh in the candidate's mind. This immediacy is not possible with conventional credentialing systems that rely on feedback from manual reviewers. The ultrasound credentialing embodiments disclosed herein are objective and unbiased in reviewing and grading ultrasound data submitted by candidates for credentialing. In contrast, conventional credentialing systems may rely exclusively on manual review and are therefore necessarily subjective and biased. The embodiments disclosed herein facilitate a credentialing process that can be standardized within and across different nursing facilities. In contrast, conventional credentialing systems are typically ad-hoc and not standardized across nursing facilities. The embodiments disclosed herein reduce the burden on resources within a care facility, such as the time demands of reviewers (e.g., trained clinicians), as compared to traditional credentialing systems.

[0021] References herein to "one embodiment," "one embodiment," "one example," or "one example" mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment. The appearances of the phrases "in one embodiment" or "in an embodiment" in various places in this specification do not necessarily all refer to the same embodiment. The processes illustrated in the following figures are performed by processing logic comprising hardware (e.g., circuits, dedicated logic, etc.), software, or a combination of both. Although the processes are described below with respect to some sequential operations, it should be understood that some of the described operations may be performed in different orders. Additionally, some operations may be performed in parallel rather than sequentially.

[0022] As used herein, the term "and / or" represents three relationships between objects that may be present: for example, A and / or B can represent the cases where only A is present, where both A and B are present, and where only B is present, and where A and B may be singular or plural.

[0023] FIG. 1 is a diagram 100 illustrating an environment for implementing a qualification system 101 according to some embodiments. The qualification system includes an ultrasound system that can be operated by a candidate 105 (e.g., a student training to become a qualified ultrasound operator). The ultrasound system 101 includes a probe 103 and a computing device 102 that is connected to the probe via a communication link. The communication link between the probe and the computing device can be wired, wireless, or a combination thereof. The computing device 102 can include any suitable computing device, such as a tablet, a smartphone, a heads-up display, goggles, glasses, an ultrasound machine, or a combination thereof. As shown in FIG. 1, the computing device 102 can include one or more neural networks (NNs) 106 that can be implemented at least in part in the hardware of the computing device. The computing device 102 also includes a candidate qualification application 107 that can be implemented by a processor (not shown in FIG. 1) of the computing device. For example, the memory of the computing device can store instructions that, when executed by the processor, cause the computing device to implement the candidate qualification application. As shown in FIG. 1, the candidate qualification application 107 is connected to a simulator system 108. In some embodiments, the simulator system 108 includes a dummy patient, and the ultrasound probe transmits ultrasound waves to the dummy patient and generates ultrasound data based on reflections of the ultrasound waves from the dummy patient, as described in more detail below.

[0024] As shown in FIG. 1 , the ultrasound system 101 is connected to a network 109 (e.g., a computing network). In some embodiments, one or more of the probe 103 and the computing device 102 are connected to the network 109 to connect the ultrasound system to the network. In one example, the network includes a care facility network, such as a secure Wi-Fi network operated by a care facility (e.g., a hospital). The credentialing system can include a server system 111 connected to the network 109, whereby the server system communicates with the computing device 102 operated by the candidate. The server system 111 can include a credentialing database for archiving data related to the credentialing of the candidate, such as test results, examiner feedback, ultrasound data, and metadata related to the ultrasound data. In one example, the server system includes a database of neural networks, and the computing device 102 operated by the candidate can include one or more neural networks provided by the server system. For example, the neural network 106 can include one or more neural networks received from the server system 111. In some embodiments, the credentialing application 107 determines the test type being performed by the candidate, and the server system searches for a neural network appropriate for the test type and provides the neural network to the credentialing application for implementation on the candidate's computing device. In some embodiments, the candidate's computing device includes multiple computing devices. In some embodiments, the candidate's computing device includes a server system and the neural network implemented "in the cloud" by the server. Alternatively, the server system may be separate from the candidate's computing device, such as a server system maintained by a care facility.

[0025] The credentialing system may also include an examiner computing device 113 operated by an examiner 112, such as a trained and certified sonographer, to manually review and grade ultrasound data submitted by a candidate 105, for example, as part of a credentialing process for the candidate. The examiner computing device 113 may include an examiner credentialing application 114, which may be executed by a processor (not shown in FIG. 1 ) of the examiner computing device. For example, a memory of the examiner computing device may store instructions that, when executed by the processor, cause the examiner computing device to perform the examiner credentialing application. The examiner, via a user interface of the examiner credentialing application, may perform any suitable function as part of the credentialing review, such as reviewing the ultrasound data supplied by the candidate, providing feedback (e.g., comments, grading, etc.) back to the candidate, and archiving the data on a server system.

[0026] During an ultrasound examination of a patient 104, an ultrasound examination candidate 105 can use an ultrasound system 101 (e.g., a probe 103 and a computing device 102) to generate ultrasound data for credentialing by a credentialing system. In one example, unlike conventional credentialing systems, the credentialing system does not necessarily transmit the ultrasound data to a reviewer for manual review, but instead implements one or more neural networks 106, such as on the candidate's computing device 102, to automatically determine characteristics of the ultrasound data and provide feedback to the candidate 105 based on the characteristics in real time during the ultrasound examination. Feedback generated by the neural networks can also be communicated by the computing device 102 to a server system 111 for archiving as part of the credentialing.

[0027] The neural network 106 may generate any suitable output based on the ultrasound data. In some embodiments, the qualification system implements the neural network to generate a quality indicator for the ultrasound image, such as a number between 0 and 1, where 0 indicates poor quality and 1 indicates excellent quality, or a binary label, such as "pass" or "fail." In some embodiments, the binary label is generated by applying a threshold to the probability of the indicator generated by the neural network, such as the probability that the ultrasound image contains an acceptable view. In some embodiments, the threshold is an image quality threshold, or other indicator threshold. In some embodiments, the image quality indicator indicates that the image has sufficient quality to perform a predetermined operation. In some embodiments, the neural network is trained to generate the binary label directly (e.g., without first generating probabilities and applying a threshold thereto) by appropriate selection of the loss function used to train the neural network. The candidate qualification application 107 may display the quality indicator to the candidate via a user interface of the computing device 102. Additionally or alternatively, the candidate qualification application 107 may generate a grade based on the quality of the ultrasound image, such as by assigning a letter grade A, B, C, D, or F based on the quality index generated by the neural network 106. The grade may be an indication of the usefulness of the ultrasound data, such as whether an ultrasound image generated from the ultrasound data is "good enough" to determine a pneumothorax condition.

[0028] The neural network 106 may receive any suitable input to generate a quality index and / or grade for the candidate. As previously described, the neural network may receive one or more ultrasound images generated during an ultrasound examination by the candidate 105. Additionally, the neural network may include one or more secondary (e.g., additional) inputs for generating a quality index and / or grade for the candidate, as described in further detail with respect to FIG.

[0029] In some embodiments, the ultrasound probe 103 includes an inertial measurement unit (IMU) capable of measuring one or more of force, acceleration, angular velocity, and magnetic field. The IMU may include a combination of accelerometers, gyroscopes, and magnetometers and may generate position and / or orientation data including data representative of six degrees of freedom, such as yaw, pitch, and roll angles in a coordinate system. Additionally or alternatively, the ultrasound system may include a camera for determining position and / or orientation data of the ultrasound probe. The position and / or orientation data may be indicative of movement of the probe. The neural network may process the position and / or orientation data as a secondary input in addition to the ultrasound image.

[0030] In some embodiments, the neural network also receives as a secondary input an indication of the candidate's behavior during the test. For example, an unskilled or untrained ultrasound operator may move the probe more than a skilled or trained operator during an ultrasound test, which may cause discomfort to the patient. Thus, one example of a secondary input indicative of the candidate's behavior includes the candidate's motion data. For example, the candidate may wear a motion sensor on their clothing that determines the candidate's motion data during the test (e.g., the extent of the candidate's arm movement). In another example, the probe includes one or more motion sensors (e.g., IMU, gyro, position / orientation sensor, movement sensor, or other motion sensor) to collect the candidate's motion data. Additionally or alternatively, a camera in the examination room may generate the motion data. In one example, a secondary input to the neural network includes an amount of time, such as the time it takes the candidate to generate and save the ultrasound image for review, starting from when the candidate begins to acquire an ultrasound image using the probe / ultrasound system. For example, an "unskilled" operator may take a lot of time to move the probe to the proper location to obtain a proper ultrasound image, and this excessive time may have an adverse effect on the patient.

[0031] Additionally or alternatively, the secondary input indicative of the candidate's behavior can include audio content, including audio content spoken by the candidate, the patient, and / or another person viewing the test. For example, the audio content can include a conversation between the candidate and the patient, such that the neural network can be trained to generate a better score for the candidate when there is positive communication with the patient, such as by telling the patient what to expect next during the test. In one example, the patient says "You're hurting me" or someone in the room tells the candidate to move the probe in a particular way (e.g., the candidate gets help to get a proper view). FIG. 2 is a diagram 200 illustrating an ultrasound probe 201 having a sensor region 205 for determining grip orientation according to some embodiments. In some embodiments, the ultrasound probe 201 corresponds to the ultrasound probe 103 or another ultrasound probe. In some embodiments, the ultrasonic probe 201 corresponds to an ultrasonic probe having a sensor area for determining grip orientation, as described in U.S. Patent Application No. 18 / 045,477, entitled "CONFIGURING ULTRASOUND SYSTEMS BASED ON SCANNER GRIP," filed October 11, 2022, attorney docket number 105767P025, which is incorporated herein by reference in its entirety. As shown in FIG. 2, the ultrasonic probe 201 includes a sensor area 205 for detecting grip orientation and a transducer connected to a lens 203. In FIG. 2, the sensor area 205 is shown as an ellipsoid. However, the sensor area 205 can be any suitable shape. In some embodiments, the sensor area 205 substantially covers a surface of the ultrasonic probe 201, for example, the sensor area can cover substantially all of the ultrasonic probe 201, including or excluding the lens 203. In some embodiments, the sensor area 205 covers the area that is typically gripped by a user (eg, the grip of the probe, which is narrower than the transducer head of the probe).

[0032] In some embodiments, the transducer of the ultrasonic probe 201 includes an ultrasonic transducer array and electronics coupled to the ultrasonic transducer array to transmit ultrasonic signals to the patient's anatomy and receive ultrasonic signals reflected from the patient's anatomy. The ultrasonic probe 201 can include any suitable type of sensor in, on, or under the sensor area 205 to determine grip orientation. In some embodiments, the ultrasonic probe 201 includes a capacitance sensor that can measure capacitance or a change in capacitance caused by a user's touch or proximity of a touch, as is common in touch screen technology. Additionally or alternatively, the ultrasonic probe 201 can include a pressure sensor configured to determine the amount of pressure caused by a user gripping the probe.

[0033] In some embodiments, the ultrasound system receives sensor data from sensors in the sensor region 205 and generates a grip map 207 representing the sensor data. As shown in FIG. 2, the grip map 207 includes a two-dimensional (2D) grid (e.g., a matrix) of sensor data. The nodes of the grid can correspond to sensors in the sensor region 205 and can include sensor data from the sensors. As an example, each intersection of the cross-hatching shown in the sensor region 205 of FIG. 2 can correspond to a sensor for determining a grip orientation and thus a node in the 2D grid. The sensor data can include a multi-level indicator indicating a magnitude of pressure on the sensor, such as on an integer scale of 0 to 5. For example, a "0" can indicate that no pressure from the user's hand is detected by the sensor, and a "1" can indicate that a small amount of pressure from the user's hand is detected by the sensor. A "2" can indicate that an amount of pressure from the user's hand greater than a "1" is detected by the sensor, and a "5" can indicate that a maximum amount of pressure from the user's hand is detected by the sensor. A neural network (e.g., NN 106 of FIG. 1) may receive grip orientation, such as grip map 207, as a secondary input in addition to the ultrasound image to generate a quality index and / or grade for the candidate. Other examples of secondary inputs include pressure data (e.g., how much pressure is applied from the probe to the patient), time data (e.g., how long it takes the candidate to operate the ultrasound system to perform an ultrasound exam), etc.

[0034] FIG. 3 is a diagram 300 illustrating a system for generating data related to a candidate's evaluation based on one or more ultrasound images and one or more secondary inputs, according to some embodiments. As shown in FIG. 3, a neural network 303 receives an ultrasound image 301 as a primary input and one or more secondary inputs 305, as described above, to generate an output 309. The neural network 303 is an example of the neural network 106 of FIG. 1. In some embodiments, the output 309 includes data related to the candidate's evaluation, such as the candidate's quality indicator, grade, and / or score during the ultrasound examination. The neural network 303 may combine the ultrasound image 301 with the one or more secondary inputs 305 in any suitable manner. In some embodiments, the neural network concatenates the secondary inputs and the ultrasound image and processes the concatenated data at a top (first) layer of the neural network. In some embodiments, the neural network processes the ultrasound image in one or more layers of the neural network and concatenates the results (e.g., feature maps based on the ultrasound image) with secondary inputs for subsequent layers of the neural network.

[0035] In some embodiments, the neural network 303 includes multiple networks and / or sections. Each section may include one or more neural networks. In some embodiments, the neural network processes the ultrasound images using two sections, with the second section receiving a secondary input. The neural network 303 may combine one or more of the results output from the first and second sections with the ultrasound images and one or more of the secondary inputs. In some embodiments, one or more ultrasound images 301 are input to a first neural network, and the output of the first neural network (e.g., feature maps, or other outputs) is combined with one or more secondary inputs 305 and input to a second neural network to generate an output 309. In some embodiments, one or more secondary inputs 305 are input to a first neural network, and the output of the first neural network is combined with one or more ultrasound images 301 and input to a second neural network to generate an output 309.

[0036] In some embodiments, the neural network 303 receives a weight vector that assigns relative weights to the secondary inputs 305. For example, the weight vector can include user-assigned values ​​between 0 and 1 to place more or less emphasis on the secondary inputs, for example, by weighting probe orientation data more heavily than audio data. By allowing the weights to include user-assigned weights, the qualification system can adjust to match current trends in ultrasound testing over time. For example, currently, the state of the art may place a significant emphasis on temporal data (e.g., the time it takes a candidate to perform an ultrasound test). In the future, the current state of the art may change and place more emphasis on pressure data. The qualification system can easily accommodate these changes via the weight vector. In some embodiments, the qualification system qualifies candidates based on a combination of automated review by the neural network 303 and manual review performed by a reviewer (e.g., reviewer 112 of FIG. 1). For example, the credentialing system may limit review of ultrasound data to the candidate credentialing application and automated review of ultrasound data using the neural network described above until a competency threshold is met, at which point the credentialing system may provide ultrasound data from the candidate to a reviewer for review as part of a manual review.

[0037] An example of a competency threshold is a candidate receiving a passing score (e.g., a binary "pass" label or a letter grade of A or B) for a specified number of tests, such as the last five tests. Additionally or alternatively, a competency threshold may require a candidate to pass certain tests (e.g., receive an acceptable letter grade), such as for a bladder scan or a scan with an ultrasound protocol. An example of an ultrasound protocol is the Extended Focused Assessment with Ultrasound in Trauma (eFAST), which is designed to detect ascites, pericardial effusion, pneumothorax, and / or hemothorax in trauma patients. In one example, for a vascular ultrasound examination of the lower extremities to diagnose the presence or absence of deep vein thrombosis (DVT), the qualification system may evaluate whether several reference sections are observed consecutively from the central side to the peripheral side. In another example, for an abdominal examination, if the order of observing multiple representative sites is predetermined for each hospital, the qualification system may evaluate whether the examination was performed according to the predetermined order. Thus, the qualification system may transfer a candidate from an automated review to a manual review based on the history of tests performed by the candidate.

[0038] In some embodiments, to qualify a candidate, the qualification system subjects the candidate to several screening iterations, each iteration including an initial automated review by a neural network followed by a manual review by a human reviewer upon successful completion of the automated review. In some embodiments, the qualification system groups the iterations of the screening process based on one or more characteristics, such as by difficulty, content, patient availability, and other characteristics. For example, a first iteration may include an ultrasound examination of the bladder, a second iteration may include an ultrasound examination with an ultrasound protocol, a third iteration may include an ultrasound examination of the cardiac anatomy, a fourth iteration may include an ultrasound examination using a simulator system (e.g., simulator system 108) (described in more detail below), etc.

[0039] In some embodiments, the credentialing system includes a credentialing database that stores the test results and test data of ultrasound candidates. For example, the credentialing database can store data used as input to the neural network and data generated by the neural network as shown in FIG. 3, such as ultrasound images, sensor data, motion data, orientation data, voice spoken by the candidate and / or patient, grip map, test score, image quality score, etc. The server system 111 of FIG. 1 can include a credentialing database. The data stored in the credentialing database can be provided as input to the neural network. Thus, the neural network can generate a test score for an ultrasound candidate based on data from past tests performed by the candidate as well as data from a current test performed by the candidate. The credentialing system can be configured to weight previous tests differently from current tests, such as, for example, using a more difficult threshold (e.g., a higher score threshold) for the current test and easier thresholds for previous tests. In one example, data from the credentialing database is provided to a decision tree that deterministically generates a test score based on the history of the ultrasound candidate's data. Additionally or alternatively, data from the credentialing database may be provided to a neural network trained to generate exam scores from the ultrasound exam candidate's historical data.

[0040] FIG. 4 is a data flow diagram of a process 400 implemented by a credentialing system for issuing a sonographer credential to a sonography candidate in some embodiments. With reference to FIGS. 1 and 4, the credentialing system includes an ultrasound system including a probe 103 connected to a computing device 102. In some embodiments, the computing device includes one or more processors and memory connected to the processor for executing the process 400. In some embodiments, the process 400 is executed using processing logic. The processing logic may include hardware (circuitry, dedicated logic, etc.), software (such as running on a general purpose computer system or a dedicated machine), firmware, or a combination thereof. In block 401, ultrasound data is generated using an ultrasound probe. In block 402, a sonography score is generated as part of an automated review based on the ultrasound data. In some embodiments, the computing device is configured to implement one or more neural networks 106 to generate an image quality score based on the ultrasound data as part of the automated review, and the sonography score is based on the image quality score. In some embodiments, the computing device is implemented to determine an anatomical structure based on the ultrasound data. The computing device can then select a neural network from among the multiple neural networks based on the anatomical structure.

[0041] In some embodiments, the ultrasound probe includes a touch-sensitive surface, and the processor is implemented to generate a grip orientation on the touch-sensitive surface. The processor may represent the grip orientation as a grip map, as described above. The computing device is implemented to generate an ultrasound examination score based on the grip orientation. In some embodiments, the ultrasound probe includes a pressure sensor implemented to generate pressure data indicative of a magnitude of pressure of the ultrasound probe on the patient, and the computing device is implemented to generate an ultrasound examination score based on the pressure data. In some embodiments, the qualification system includes an audio processor implemented to record audio content, and the computing device is implemented to generate an ultrasound examination score based on the audio content.

[0042] In some embodiments, the ultrasound probe includes an inertial measurement unit implemented to generate motion data of the ultrasound probe, and the computing device is implemented to generate an ultrasound examination score based on the motion data. In some embodiments, the qualification system includes a sensor system implemented to generate motion data of the ultrasound examination candidate (e.g., data indicative of how the ultrasound examination candidate moves during the examination), and the computing device is implemented to generate an ultrasound examination score based on the motion data. In some embodiments, the sensor system includes a wearable sensor configured to be worn by the ultrasound examination candidate. The motion data can be generated based on data sensed by the wearable sensor. In some embodiments, the qualification system includes a simulator system (e.g., simulator system 108 of FIG. 1). The simulator system can include a dummy patient, and the ultrasound probe can be implemented to transmit ultrasound to the dummy patient and generate ultrasound data based on reflection of the ultrasound from the dummy patient. As shown in FIG. 4, the computing device transfers data related to the ultrasound examination candidate from the automated review to a manual review by a reviewer based on the ultrasound examination score in block 403. By transferring ultrasound candidates from neural network-based automated review to reviewer-based manual review, the computing device improves ultrasound candidates through the credentialing process to move them closer to the goal of issuing a ultrasound certification. In some embodiments, the credentialing system anonymizes the candidate's data when the candidate is transferred from automated to manual review, so that reviewers cannot identify the candidate. This anonymization can remove any inherent bias by reviewers.

[0043] In some embodiments, the qualification system includes a qualification database that stores the test results of the ultrasound candidates, as described above. The computing device can transfer the ultrasound candidates from the automated screening to the manual screening based on the test results from the qualification database for at least some of the ultrasound candidates that are different from the ultrasound candidates. In some embodiments, the qualification system is trained to look at the ultrasound candidates that passed the manual screening and what they have in common with each other, such as what type of tests they passed in the automated screening. The qualification system can then transfer data related to the current ultrasound candidate from the automated screening to the manual screening only if the current ultrasound candidate also passed the tests in the automated screening that the other ultrasound candidate passed. In this way, the qualification system can use the data of the other ultrasound candidates as a predictor of the performance of the current ultrasound candidate. The data from the qualification database can be provided to a neural network that is trained to predict the performance of the current ultrasound candidate. The neural network can be a neural network as shown in FIG. 1 or can be an additional neural network.

[0044] In some embodiments, the credentialing system uses data for multiple ultrasound candidates (e.g., data from a credentialing database) to determine a test score, such as a grade, for the current candidate. For example, the credentialing system may look at multiple test scores for multiple candidates and grade them "on a curve" according to their test scores, such as by assigning a passing grade to the top half of the candidates and a failing grade to the bottom half of the candidates, or by assigning a passing grade to the top 30% of the candidates and a failing grade to the bottom 70% of the candidates.

[0045] FIG. 5 is a data flow diagram of a process 500 implemented by a credentialing system for issuing a sonographer credential to an ultrasound examination candidate in some embodiments. With reference to FIGS. 1 and 5, the credentialing system includes an ultrasound system 101 configured to generate ultrasound images (block 501). The credentialing system includes a candidate credentialing application 107 implemented at least in part in the credentialing system hardware and configured to generate an image quality score for a first subset of the ultrasound images (block 502) and communicate a second subset of the ultrasound images to an examiner computing device based on the image quality score (block 503). For example, the first subset of images can correspond to a first anatomical structure being imaged (e.g., bladder), a first ultrasound protocol (e.g., eFAST), a first exam preset (e.g., bladder preset), etc., and the second subset of images can correspond to a second anatomical structure being imaged (e.g., lungs), a second protocol (e.g., a protocol other than eFAST), a second exam type (e.g., cardiac preset), etc.

[0046] In some embodiments, the first subset of ultrasound images and the second subset of ultrasound images are disjoint. In some embodiments, the qualification system includes an examiner computing device 113 connected to the computing device 102. In some embodiments, the candidate qualification application determines guidance for the ultrasound candidate to improve at least one of the image quality scores based on at least one of the image quality scores being below a threshold score. The qualification system can include a display device that displays the guidance in a visual representation. In some embodiments, the visual representation includes at least one of a training video, an icon of an ultrasound probe, an arrow indicating a direction to move the ultrasound probe, and an icon of a grip orientation for holding the ultrasound probe.

[0047] In some embodiments, the candidate qualification application communicates the second subset of ultrasound images to the examiner computing device based on at least one of the percentage of image quality scores being above a threshold score. For example, the candidate qualification application communicates the second subset of ultrasound images if the percentage of image quality scores is greater than the threshold score. In some embodiments, the first subset includes at least one ultrasound image of a particular anatomical structure. In some embodiments, the first subset includes at least one ultrasound image generated with an ultrasound system in a specified imaging mode. In some embodiments, the first subset includes at least one ultrasound image generated according to a specified examination protocol.

[0048] In some embodiments, the candidate qualification application includes a neural network that generates an image quality score based on at least one of a grip orientation of an ultrasound probe of an ultrasound system (e.g., a grip map as described above with respect to FIG. 2), an amount of movement of the ultrasound probe, an amount of movement of the ultrasound candidate, audio content, an amount of pressure applied to the patient by the ultrasound probe, and an amount of time taken by the ultrasound candidate to operate the ultrasound system to generate an ultrasound image.

[0049] FIG. 6 is a data flow diagram of a process 600 implemented by a ultrasound examination qualification system according to some embodiments. With reference to FIGS. 1 and 6, the ultrasound examination qualification system includes an ultrasound probe 103 configured to generate ultrasound data (block 601). The ultrasound examination qualification system includes a computing device 102 configured to generate ultrasound images based on the ultrasound data (block 602). The ultrasound examination qualification system includes a neural network 106 implemented at least in part in the hardware of the computing device to generate an image quality score based on the ultrasound images (block 603). The ultrasound examination qualification system includes a qualification device (e.g., implemented at least in part by a processor of the computing device) configured to issue a sonographer qualification based on the image quality score (block 604).

[0050] In some embodiments, the ultrasound credentialing system issues a certificate based on the candidate's score history. In some embodiments, the ultrasound credentialing system issues a certificate if the candidate's score is greater than 2 / 3 of the other candidates' scores. In some embodiments, the ultrasound credentialing system issues a certificate when the candidate is in the top third of all candidates or other criteria.

[0051] In some embodiments, the qualification system provides guidance to the candidate during the ultrasound examination. For example, the guidance may include hints displayed by the candidate qualification application on a user interface of the computing device. The qualification system may determine the guidance using one or more of the neural networks. For example, the neural network may determine, based on the ultrasound data (e.g., ultrasound images), that imaging parameters need to be adjusted to improve the quality of the ultrasound images. Examples of imaging parameters include gain, depth, and exam type. The neural network may generate adjustments to the imaging parameters, and the candidate qualification application may display a message on the user interface to adjust the imaging parameters according to the recommended adjustments from the neural network.

[0052] In some embodiments, the guidance includes an indication of the movement of the probe. The qualification system communicates the guidance via a user interface, for example, by displaying directional arrows, broadcasting a voice recommendation "move the probe towards the center of the patient", haptic feedback on the probe, combinations thereof, etc. In some embodiments, the guidance includes a recommendation of a particular training to the candidate. For example, the qualification system can determine training material from a database of training materials based on one or more of the neural network output, the type of ultrasound exam being performed, the anatomy being imaged, the imaging parameters, and the ultrasound data to improve the candidate's ultrasound examination skills. A server system can maintain the database of training materials and provide the recommended training materials to the candidate qualification application upon request from the candidate computing device.

[0053] In some embodiments, the credentialing system generates and communicates guidance to the candidate computing system based on a request (e.g., a candidate request) provided by the candidate to the credentialing system. The candidate request can be spoken, typed, gestured, etc. For example, the candidate can speak, "Help, how do I hold the probe?" or gesture with the probe in a specified manner, such as swiping an "X" in the air, to indicate that help is needed. In some embodiments, the credentialing system can generate guidance for a candidate without an explicit request from the candidate. For example, the credentialing system can determine from the output of a neural network in a computing device that the candidate has not passed an ultrasound examination (e.g., the neural network has generated a quality metric that corresponds to a fail score). Thus, the credentialing system can communicate guidance to the candidate instructing the candidate on proper use of the ultrasound system. In some embodiments, if the credentialing system communicates guidance to the candidate of the current ultrasound data, the credentialing system does not accept the current ultrasound data to credential the candidate. Rather, the credentialing system requires candidates to generate new ultrasound data without guidance being provided for the new ultrasound data and to submit this new ultrasound data for consideration toward credentialing.

[0054] Referring back to FIG. 1 , the qualification system includes a simulator system 108 connected to a computing device operated by the candidate 105. The simulator system 108 includes hardware, software, firmware, or a combination thereof for emulating an ultrasound examination of a live patient when a live patient is not available. For example, for some types of ultrasound examinations and patient conditions that require immediate attention, such as detection of free fluid, it may not be feasible for the candidate to approach a live patient in that condition to perform the ultrasound examination in training for the purpose of qualification, since such conditions may be life-threatening. Thus, the qualification system includes a simulator system capable of generating image data. For example, the qualification system may include a dummy patient (e.g., torso and head, full body dummy, etc.) that can be imaged by an ultrasound probe. The dummy patient may include an artificial anatomical structure, and the ultrasound probe and the computing device may generate an ultrasound image of the artificial anatomical structure.

[0055] In some embodiments, the simulator system generates image data that mimics ultrasound images but is not directly derived from ultrasound signals transmitted by the ultrasound system. For example, the image data may be generated by the simulator system based on position and orientation data of the probe, such as the contact points or contact areas on a dummy patient, and data corresponding to the six degrees of freedom of the probe. In some embodiments, the simulator system generates image data based on imaging parameters set by the candidate. The position data, orientation data, and imaging parameters may be aggregated, for example, into a vector and provided as input to the neural network 106. The neural network may be trained to generate images that look like ultrasound images, which may be used by the credentialing system to authenticate the candidate.

[0056] In some embodiments, the images generated by the simulator system 108 are reviewed by one or more neural networks of the candidate's computing device 102, as described above as part of the automated review. For example, the neural network can generate a quality metric for the image, or a grade for the image. In some embodiments, the qualification system determines the quality metric or grade for the image based on ground truth images. For example, the qualification system can include a database of ground truth images collected by trained experts for various anatomical structures and exam types. The qualification system can compare the images generated by the simulator system to the ground truth images, such as mean squared error (e.g., pixels or features extracted from the image), to determine the quality metric or grade for the image. In some embodiments, the neural network receives ground truth images as a second (or conditional) input in addition to the images generated by the simulator system to generate a quality metric or grade for the images generated by the simulator system.

[0057] FIG. 7 is a data flow diagram of a method 700 implemented by a computing device according to some embodiments, such as the computing device 102 of FIG. 1. The method 700 includes, at block 701, receiving ultrasound data from an ultrasound probe connected to the computing device. In some embodiments, the ultrasound probe is instructed by the computing device to transmit an ultrasound signal to a dummy patient and determine ultrasound data based on the ultrasound signal. At block 702, an image quality score is generated by the computing device based on the ultrasound data. In some embodiments, the image quality score is generated using a neural network implemented at least in part in the hardware of the computing device. In some embodiments, an anatomical structure is determined based on the ultrasound data, and the neural network is selected from among a plurality of neural networks based on the anatomical structure. At block 703, the ultrasound data is communicated to an examiner computing device for user qualification based on the image quality score.

[0058] In some embodiments, additional ultrasound data is received from the ultrasound probe, and an additional image quality score is generated by the computing device based on the additional ultrasound data. In some embodiments, the computing device determines not to communicate the additional ultrasound data to the examiner computing device for user qualification based on the additional image quality score. In some embodiments, guidance for improving the additional image quality score is displayed on a user interface of the computing device. In some embodiments, the guidance includes at least one of instructions to move the probe, adjustment of imaging parameters, and exam type. In some embodiments, training material is selected based on the additional image quality score and the additional ultrasound data. In some embodiments, the training material is published on the computing device for utilization by the user.

[0059] FIG. 8 is a data flow diagram of a method 800 implemented by a computing device according to some embodiments, such as the computing device 102 of FIG. 1. The method 800 includes, at block 801, receiving ultrasound data from an ultrasound probe connected to the computing device. At block 802, an image quality score is generated by the computing device based on the ultrasound data. At block 803, based on the image quality score, communication of the ultrasound data to the assessor computing device for user qualification is stopped. Because the image quality score is below a threshold quality score, the computing device may stop communication of the ultrasound data. Thus, when the qualification system determines that the ultrasound data is not "good enough," the assessor spends time evaluating the ultrasound data as part of a manual review. Thus, the qualification system is more efficient than conventional qualification systems that rely solely on manual review and may waste assessor time.

[0060] Figure 9 is a data flow diagram of a method 900 implemented by a computing device according to some embodiments, such as the computing device 102 of Figure 1. The method 900 includes receiving ultrasound data from an ultrasound probe connected to the computing device, at block 901. At block 902, an image quality score is generated by the computing device based on the ultrasound data. At block 903, the image quality score and the ultrasound data are communicated to a certification server for user certification.

[0061] FIG. 10 is a data flow diagram of a method 1000 implemented by a computing device according to some embodiments, such as the computing device 102 of FIG. 1. The method 1000 includes receiving ultrasound images at block 1001. At block 1002, an image quality score is generated for a first subset of the ultrasound images. At block 1003, a second subset of the ultrasound images is communicated to an examiner computing device for user qualification based on the image quality score. In some embodiments, the first subset of ultrasound images and the second subset of ultrasound images are distinct from one another. For example, the first subset of images can correspond to a first anatomical structure being imaged (e.g., bladder), a first ultrasound protocol (e.g., eFAST), a first exam preset (e.g., bladder preset), etc., and the second subset of images can correspond to a second anatomical structure being imaged (e.g., lungs), a second protocol (e.g., a protocol other than eFAST), a second exam type (e.g., cardiac preset), etc.

[0062] In some embodiments, the method 1000 includes communicating the first subset of ultrasound images and the image quality score to a credentialing server for credentialing of the user. In some embodiments, the method 1000 includes communicating a comment to a candidate computing device that generated the ultrasound images. In some embodiments, the comment indicates communicating the second subset to an examiner computing device. In some embodiments, the comment includes at least one of the image quality scores of the first subset. In some embodiments, the comment indicates that at least one ultrasound image of the first subset has an image quality score corresponding to a failing grade. In some embodiments, the method 1000 includes obtaining a number of image thresholds. In some embodiments, the comment indicates that a number of image thresholds of at least the first subset of ultrasound images has an image quality score corresponding to a passing grade.

[0063] 11 is a data flow diagram of a method 1100 implemented by a computing device according to some embodiments, such as computing device 102 and / or examiner computing device 113 of FIG. 1. Method 1100 includes receiving an ultrasound image at block 1101. At block 1102, an image quality score for the ultrasound image is generated using a neural network implemented at least in part in the hardware of the computing device. At block 1103, an ultrasound certification is issued based on the image quality score.

[0064] FIG. 12 is a block diagram of an exemplary computing device 1200 capable of performing one or more of the operations described herein, according to some embodiments. The computing device 1200 may be connected to other computing devices in a LAN, an intranet, an extranet, and / or the Internet. The computing device may operate in the capacity of a server machine in a client-server network environment or in the capacity of a client in a peer-to-peer network environment. The computing device may be provided by a personal computer (PC), a server computer, a desktop computer, a laptop computer, a tablet computer, a smartphone, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the machine. Furthermore, although only a single computing device is shown, the term "computing device" should also be construed to include any collection of computing devices that individually or together execute a set (or sets) of instructions to perform the methods described herein. In some embodiments, the computing device 1200 may be one or more of an access point and a packet forwarding component.

[0065] The exemplary computing device 1200 may include a processing device (e.g., a general purpose processor, PLD, etc.) 1202, a main memory 1204 (e.g., synchronous dynamic random access memory (DRAM), read only memory (ROM)), and a static memory 1206 (e.g., flash memory and data storage device 1218), which may communicate with each other via a bus 1230. The processing device 1202 may be provided by one or more general purpose processing devices, such as a microprocessor, a central processing device, etc. In an exemplary example, the processing device 1202 may comprise a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or a processor implementing a combination of instruction sets. The processing device 1202 may also comprise one or more special purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. The processing device 1202 may be configured to perform the operations and steps described herein in accordance with one or more aspects of the present disclosure.

[0066] The computing device 1200 may further include a network interface device 1208 capable of communicating with a network 1220. The computing device 1200 may also include a video display unit 1210 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 1212 (e.g., a keyboard), a cursor control device 1214 (e.g., a mouse), and an audio signal generating device 1216 (e.g., a speaker and / or a microphone). In one embodiment, the video display unit 1210, the alphanumeric input device 1212, and the cursor control device 1214 may be combined into a single component or device (e.g., an LCD touch screen).

[0067] The data storage device 1218 may include a computer-readable storage medium 1228 that may store one or more sets of instructions 1226, such as instructions for performing operations described herein, according to one or more aspects of the present disclosure. Additionally, the instructions 1226 may reside, completely or at least partially, within the main memory 1204 and / or within the processing device 1202 during execution by the computing device 1200, the main memory 1204, and the processing device 1202, which also constitute computer-readable media. Additionally, the instructions may be transmitted or received over a network 1220 via the network interface device 1208.

[0068] Although the computer-readable storage medium 1228 is shown in the illustrative example as being a single medium, the term "computer-readable storage medium" should be interpreted to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) that store one or more sets of instructions. The term "computer-readable storage medium" should also be interpreted to include any medium that can store, encode, or carry a set of instructions for execution by a machine and cause the machine to perform the methods described herein. Thus, the term "computer-readable storage medium" should be interpreted to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0069] Unless otherwise specified, terms such as "transmit," "determine," "receive," "generate," and the like refer to acts and processes performed or implemented by a computing device that manipulates data represented as physical (electronic) quantities in the registers and memory of the computing device to convert it into other data that is similarly represented as physical quantities in the memory or registers of the computing device or other such information storage, transmission, or display device. Also, terms such as "first," "second," "third," "fourth," and the like, as used herein, are intended as labels to distinguish between different elements and do not necessarily imply an order according to their numerical designations.

[0070] The examples described herein also relate to apparatus for performing the operations described herein. This apparatus may be specially constructed for the required purposes, or may comprise a general-purpose computing device selectively programmed by a computer program stored in the computing device. Such a computer program may be stored in a computer-readable non-transitory storage medium.

[0071] The methods and illustrative examples described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems can be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems appears as described in the description above.

[0072] The above description is illustrative and not restrictive. Although the present disclosure has been described with reference to certain exemplary embodiments, it will be recognized that the present disclosure is not limited to the described embodiments. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which such claims are entitled.

[0073] As used herein, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly indicates otherwise. It is further understood that the terms "comprises," "comprising," "including," and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Accordingly, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0074] It should also be noted that in some alternative implementations, the functions / acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed substantially concurrently or may sometimes be executed in the reverse order, depending on the functions / acts involved.

[0075] Although the method operations have been described in a particular order, it should be understood that other operations may be performed between the operations described, the operations described may be coordinated so that they occur at slightly different times, or the operations described may be distributed across a system that enables the occurrence of the processing operations at various intervals relative to the process.

[0076] Various units, circuits, or other components may be described or claimed as being "configured to" or "configurable to" perform one or more tasks. In such contexts, the phrase "configured to" or "configurable to" is used to imply structure by indicating that the unit / circuit / component includes structure (e.g., circuitry) that performs one or more tasks during operation. Thus, a unit / circuit / component may be said to be configured to perform a task or to be configurable to perform a task even if the specified unit / circuit / component is not currently operating (e.g., not on). A unit / circuit / component used with the language "configured to" or "configurable to" includes hardware, e.g., circuits, memory that stores executable program instructions to perform an operation, etc. It is expressly intended that specifying that a unit / circuit / component is "configured to" perform one or more tasks or is "configurable to" perform one or more tasks does not invoke 35 U.S.C. 112, paragraph 6, for that unit / circuit / component. Additionally, "configured to" or "configurable to" can include general structure (e.g., general purpose circuitry) that is manipulated by software and / or firmware (e.g., an FPGA or general purpose processor executing software) to operate in a manner that can perform the tasks in question. "Configured to" can also include adapting a manufacturing process (e.g., a semiconductor manufacturing facility) to manufacture devices (e.g., integrated circuits) adapted to perform or execute one or more tasks. It is expressly intended that "configurable to" does not apply to blank media, unprogrammed processors or unprogrammed general purpose computers, or unprogrammed programmable logic devices, programmable gate arrays, or other unprogrammed apparatus, unless accompanied by programmed media that empowers the unprogrammed apparatus to be configured to perform the disclosed functions.

[0077] The above description has been described with reference to specific embodiments for purposes of illustration. However, the above illustrative description is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been chosen and described to best explain the principles of the embodiments and their practical application, and thereby enable others skilled in the art to best utilize the embodiments and various modifications that may be adapted to the particular applications contemplated. Thus, the present embodiments should be considered as illustrative and not restrictive, and the invention should not be limited to the details given herein, but may be modified within the scope of the appended claims and their equivalents.

Claims

1. A certification system for issuing ultrasound technician certification to ultrasound technician candidates, comprising: an ultrasound probe connected to a computing device and configured to generate ultrasound data, the grip portion including a plurality of first pressure sensors for generating grip data indicative of a magnitude of pressure caused by the ultrasound examination candidate gripping the grip portion, the grip data being a grip map indicative of a magnitude of pressure applied by the ultrasound examination candidate to a plurality of locations on the grip portion, the grip portion including a second pressure sensor configured to generate pressure data indicative of a magnitude of pressure of the ultrasound probe placed on a patient; an examiner computing device connected to the computing device via a network for an examiner to perform manual examination; The computing device comprises: generating an ultrasound image based on the ultrasound data; generating an ultrasound score as part of the automated review based on the ultrasound image, the grip map, and the pressure data; Transferring the ultrasound data of the ultrasound examination candidate from the automated screening to the reviewer computing device based on the ultrasound examination score. Certification system.

2. the computing device implementing a neural network; The neural network comprises: receiving the ultrasound image as a primary input; 2. The qualification system of claim 1, further comprising: generating an image quality score based on the ultrasound images as part of the automated screening; and wherein the ultrasound examination score is based on the image quality score.

3. The computing device comprises: determining an anatomical structure based on the ultrasound data; The qualification system of claim 2 , further comprising selecting the neural network from among a plurality of neural networks based on the anatomical structure.

4. 3. The qualification system of claim 2, wherein the grip portion includes a touch-sensitive surface, the ultrasound probe further comprising a processor that generates grip orientations as the grip map on the touch-sensitive surface, and the computing device generates the ultrasound examination score based on the grip orientations provided to the neural network as a secondary input.

5. 3. The qualification system of claim 2, further comprising an audio processor for recording audio content, the computing device generating the ultrasound examination score based on the audio content provided to the neural network as a secondary input.

6. 3. The qualification system of claim 2, wherein the ultrasound probe includes an inertial measurement unit that generates motion data on the ultrasound probe, and the computing device generates the ultrasound examination score based on the motion data provided to the neural network as a secondary input.

7. The qualification system of claim 2 , further comprising a sensor system that generates motion data on the ultrasound examination candidate, the computing device generating the ultrasound examination score based on the motion data.

8. The qualification system of claim 7 , wherein the sensor system includes a wearable sensor worn by the ultrasound examination candidate.

9. 2. The qualification system of claim 1, further comprising a simulator system including a dummy patient, wherein the ultrasound probe transmits ultrasound to the dummy patient and generates the ultrasound data based on reflection of the ultrasound from the dummy patient.

10. The qualification system of claim 1, further comprising a qualification database storing test results of ultrasound examination candidates, the computing device transferring the ultrasound data of the ultrasound examination candidate to the examiner computing device based on the ultrasound examination scores of the test results in the qualification database for at least a portion of ultrasound examination candidates different from the ultrasound examination candidate.

11. The qualification system of claim 1, wherein the computing device is configured to implement a neural network including a first input portion for receiving one or more ultrasound images, one or more second input portions for receiving additional data including the grip data, and an output portion connected to the first input portion and the second input portion and generating the ultrasound examination score.

12. The certification system of claim 11, wherein the additional data includes one or more of the candidate's motion data, voice data, probe direction data, pressure data, or time data.

13. The qualification system of claim 11, wherein the neural network includes a plurality of neural networks.

14. A qualification system as described in claim 1, comprising a camera connected to the computing device and determining the position of the ultrasound probe.

15. A computer-implemented method comprising: The method comprises: generating ultrasound data with an ultrasound probe having a grip portion, the ultrasound probe including a pressure sensor for generating pressure data indicative of a magnitude of pressure of the ultrasound probe against a patient; generating grip data by a processor indicative of a magnitude of pressure exerted by a grip of a candidate for ultrasound examination on the grip portion of the ultrasound probe, the grip data being a grip map indicative of a magnitude of pressure exerted by the candidate for ultrasound examination at a plurality of locations on the grip portion; generating an ultrasound image by the processor based on the ultrasound data; generating, by the processor, an ultrasound score based on the ultrasound image, the grip map, and the pressure data as part of an automated review; and transferring the ultrasound data of the ultrasound examination candidate to an examiner computing device for manual review by an examiner based on the ultrasound examination score.

16. The method of claim 15, wherein the ultrasound inspection score is generated as part of an automated review using a neural network, the neural network including a first input portion for receiving one or more ultrasound images, one or more second input portions for receiving additional data including the grip data, and an output portion connected to the first and second input portions for generating the ultrasound inspection score, the neural network generating an image quality score based on the ultrasound data and the grip data as part of the automated review, and the ultrasound inspection score being based on the image quality score. determining an anatomical structure based on the ultrasound data; Selecting a neural network from a plurality of neural networks based on the anatomical structure 20. The method of claim 16, further comprising:

18. The method of claim 16, wherein the additional data includes one or more of candidate motion data, voice data, probe direction data, pressure data, or time data.

Images