Robotic system for performing ultrasound scans

Abstract

A robotic system for performing automated ultrasound scans of a patient's body part includes a support surface for supporting the body part, a display, and a robotic arm. The robotic arm is configured to hold an ultrasound probe and move the ultrasound probe to obtain automated ultrasound scans of the body part supported by the support surface. The display is integrated into the support surface and adapted to support at least a portion of the body part to be scanned.

Description

[Technical Field]

[0001] The present invention relates to a robotic system for performing ultrasound scans on a body part of a patient. [Background technology]

[0002] Ultrasound scanning is widely used in the medical community as a diagnostic tool, for example when scanning a fetus, diagnosing hand or foot injuries, or diseases such as rheumatoid arthritis or gout.

[0003] Most ultrasound scans are performed by trained medical professionals. However, even the most trained ultrasound technicians have difficulty performing ultrasound scans in a consistent and efficient manner. Furthermore, because there are only a limited number of trained medical professionals, the wait time to be examined by a trained medical professional can be very long. Long wait times can lead to illnesses or injuries worsening before an examination can be performed.

[0004] To overcome some of the above problems, robotic ultrasound systems have been developed that can acquire ultrasound scans in an efficient and consistent manner, which can reduce patient wait times and improve ultrasound scans.

[0005] U.S. Patent Application Publication No. 2017 / 0181725 discloses an ultrasound imaging system including a scanning assembly, a three-dimensional (3D) image acquisition device, and a controller. The scanning assembly is configured to receive a hand or foot and includes a transducer array and an acoustic coupling fluid. The 3D image acquisition device is configured to acquire a 3D image of the hand or foot. The controller is configured to automatically adjust the direction or orientation of the transducer array relative to the hand or foot based on the 3D image of the hand or foot.

[0006] EP 2514366 A1 discloses an exemplary automated ultrasound scanning system and method. The automated ultrasound scanning system includes a multi-axis robotic arm, an ultrasound scan head disposed on the multi-axis robotic arm, a control circuit for controlling the multi-axis robotic arm, a three-dimensional image capturing device, and a computer. The computer senses an object to be examined via the three-dimensional image capturing device, creates a three-dimensional shape of the object to be examined, and plans a three-dimensional scanning path according to the three-dimensional shape. According to the three-dimensional scanning path, the computer further controls the multi-axis robotic arm to perform multi-axis motion via the control circuit, thereby performing a three-dimensional scan of the object to be examined via the ultrasound scan head, and constructs an ultrasound image according to the reflected ultrasound signals received by the ultrasound scan head.

[0007] However, these systems still have drawbacks: robotic systems can sometimes be large, cumbersome, and take up too much space. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] US Patent Application Publication No. 2017 / 0181725 [Patent Document 2] European Patent Application Publication No. 2514366 Summary of the Invention

[0009] It is an object of the present invention to provide an improved robotic system that overcomes, or at least mitigates, the problems of the prior art.

[0010] In a first aspect of the present invention, this and further objects are achieved by a robotic system for performing ultrasound scanning on a body part of a patient, the system comprising: a support surface for supporting a body part; The display and a positioning device configured to hold and move the ultrasound probe to obtain an ultrasound scan of a body part supported by the support surface; the display forming at least a portion of the support surface and adapted to support at least a portion of the body part to be scanned.

[0011] As a result, a space-efficient robotic system is achieved that does not require additional space to accommodate a display. By having the display form at least a portion of the support surface, all of the possibilities and advantages associated with having a display, some of which are described herein, are achieved without the display taking up additional space.

[0012] Furthermore, by having the display form at least part of the support surface, both the medical professional and the patient are further given the possibility to interact with the screen, whereas in prior art systems the screen is only interacted with by the medical professional.

[0013] The screen supports at least part of the body part, ensuring that at least part of the body part to be scanned is reachable by the robotic arm and within the scanning volume in which the robotic arm can perform the scan.

[0014] The ultrasound scan may be a fully autonomous ultrasound scan performed without assistance from an operator. The automated ultrasound scan may be a semi-autonomous ultrasound scan performed by the positioning device in conjunction with an operator. The operator may provide input to the robotic system regarding the movement of the positioning device, such as when to start scanning, when to stop scanning and / or movement adjustments.

[0015] The patient's body may be any part of the patient's body, such as a foot, hand, leg, arm, torso, head, etc. The robotic system may perform a full body scan of the patient, or at least a scan that scans multiple body parts of the patient.

[0016] The support surface may be any surface capable of supporting a body part of a patient. In some embodiments, the support surface is a substantially horizontal surface. In some embodiments, the support surface extends in a plane that is at an angle to the horizontal. The support surface may be formed as a desk or table. The support surface may be flat.

[0017] In one embodiment, the robotic system further comprises a secondary support means. The secondary support means may be one or more support protrusions configured to support a body part such as an arm or a leg. The one or more support protrusions may be formed as one or more braces. The secondary support means may be an additional support surface. The additional support surface may be formed in a recess configured to at least partially receive a body part such as an arm or a leg. The additional support surface may be disposed in abutment with the support surface.

[0018] The display may be any device capable of displaying one or more colors and patterns. The display may be provided with a receiver, transmitter, and / or transceiver for receiving and transmitting signals via either a wired or wireless connection. The display may further comprise a display processor. A display processor is a device comprising any circuitry and / or devices suitably adapted to perform the functions described herein. The display processor may comprise a general-purpose or special-purpose programmable microprocessor such as a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic array (PLA), a field-programmable gate array (FPGA), special-purpose electronic circuitry, or the like, or a combination thereof.

[0019] The positioning device may be a robotic arm. The positioning device may be any articulated arm configured to hold and move an ultrasound probe. The robotic arm may include multiple joints connected to each other. The robotic arm may be capable of movement in three dimensions. The articulated arm may include a holder for holding the ultrasound probe. The holder may be integrated into an end piece of the robotic arm. The end piece may be the last joint of the robotic arm. Alternatively, the holder may be connectable to the end piece of the robotic arm. The robotic arm may be provided with a receiver, transmitter, and / or transceiver for receiving and transmitting signals via either a wired or wireless connection. The robotic arm may further include a robotic arm processing unit. The robotic arm processing unit is a device comprising any circuit and / or device suitably adapted to perform the functions described herein. The robotic arm processing unit may include a general-purpose or special-purpose programmable microprocessor such as a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic array (PLA), a field-programmable gate array (FPGA), special-purpose electronic circuitry, or the like, or a combination thereof. The robotic arm may be provided with a sensor. The sensor may be for collecting data regarding movement of the robotic arm, and such a sensor may be a gyroscope or an accelerometer. The sensor may be a force sensor configured to detect a force applied by the robotic arm to a patient undergoing an automated scan. The sensor may be a force sensor configured to detect a force applied by the patient to the robotic arm during a scan. The robotic arm processing unit may receive data from the sensor and adjust movement of the robotic arm based on the received data.For example, a force threshold can be set for the force applied by the robotic arm, and if the robotic arm exceeds the force threshold, the robotic arm processing unit can stop the movement of the robotic arm or move the robotic arm away from the body part being scanned to reduce the force applied by the robotic arm. Data collected by sensors provided on the robotic arm can be transmitted to a controller configured to control the movement of the robotic arm, and the controller can use the received sensor data in controlling the movement of the robotic arm.

[0020] The positioning device may be a rail system configured to hold an ultrasound probe and move the ultrasound probe to acquire an ultrasound scan. The rail system may include an ultrasound probe holder connected to rails of the rail system. The rail system may include two rails inclined relative to each other, thereby allowing movement within a plane defined by the two rails. The two rails may be perpendicular to each other. The rail system may include three rails inclined relative to each other, thereby allowing movement within a 3D space defined by the three rails. The three rails may be perpendicular to each other.

[0021] The display may be integrated into the support surface by providing a cutout or recess in the support surface that can receive the display. The display may be integrated into the support surface by fixedly connecting the display to the support surface. The display may be integrated into the support surface such that the display surface, i.e., the surface of the display that displays the pattern and / or color, is substantially flush with the support surface. The display surface may be inclined relative to the horizontal when incorporated into the support surface. The display surface may be positioned to extend in a horizontal plane when integrated into the support surface. In some embodiments, the display substantially constitutes the support surface.

[0022] The display may be adapted to support at least a portion of the body part to be scanned by providing a flat surface on which at least a portion of the body part to be scanned can rest. The display may be adapted to support the body part to be scanned, for example, if a hand is being scanned, the entire hand may be supported by the display during scanning. The display may be adapted to support at least a portion of the body part to be scanned, for example, if a hand is being scanned, the fingers or palm of the hand may be supported by the display during scanning. The display may be a flat display. The display may be a curved display. The display may be shaped to support the body part.

[0023] In some embodiments, the display is configured to display an instruction pattern.

[0024] Displaying an instruction pattern provides a clear reference for the robotic arm to use when moving the robotic arm. Additionally, the instruction pattern can ensure that the body part supported by the support surface is within the reach of the robotic arm. The instruction outline provides explicit and clear signals, which can reduce tension and stress for the patient.

[0025] The command pattern may be a contour command for the body part to be scanned. For example, the contour command may be for a hand, arm, leg, or foot. By forming the contour command as the body part to be scanned, the patient is clear about where to place their body part to be scanned. Furthermore, since the contour command is a body part contour command, the patient can ensure that their body part is placed in the correct orientation relative to the robotic arm, facilitating scanning of the body part. Furthermore, the robot can recognize the orientation and placement of the body part based on the command pattern, which can help the robotic arm identify what is currently being scanned, for example, whether the body part being scanned is the little finger or the thumb. The command pattern may be a cross, a circle, a triangle, a square, a dot, or any other geometric shape. The command pattern can instruct the patient where to place their body part to be scanned. Thus, the command pattern can instruct the desired placement of the center of the body part to be scanned. The robotic system may include data storage for storing multiple body outlines and / or command patterns. The display may be communicatively coupled to the data storage and configured to receive one of the plurality of body contours and / or instruction patterns from the data storage and display the received body contour or instruction pattern. The display may itself retrieve the body contour or instruction pattern. Alternatively, the body contour or instruction pattern is retrieved via a controller communicatively coupled to both the display and the data storage.

[0026] In one embodiment, if the command pattern is an outline of the body part to be scanned, the outline of the body part displayed may correspond to the patient, for example, if the patient is a child or an adult, or if the patient is male or female.

[0027] The patient-specific selection of the instruction pattern may ensure a better match between the displayed contour and the patient's body part, resulting in improved initial placement of the scanned body part. The display may be configured to receive patient-related input and, in response to the input, retrieve a contour corresponding to the received input from a data storage that stores multiple body contours or instruction patterns. Alternatively, the input may be provided to a controller communicatively connectable to the display and the data storage, which may then retrieve a contour from the data storage in response to the received input and transmit the retrieved contour to the display, which may display the received contour. For example, if the received patient-related input is that the body part to be scanned is an adult male's hand, the contour of a typical adult male's hand may be retrieved from the data storage and displayed on the display, whereas if the received patient-related input is that the body part to be scanned is a female child's hand, the contour of a typical female child's hand may be retrieved from the data storage and displayed on the display.

[0028] In some embodiments, the display is configured to display a message to the patient that an automated ultrasound scan is being performed.

[0029] The message may be instructions to the patient regarding hand placement. The message may be regarding the progress of the automated scan, such as what is being scanned, the time remaining in the scan, or the action being performed by the robotic system. The message may be one or more instructions to the patient to move at least a portion of the body part supported by the display to correct the placement of at least a portion of the body part supported by the display. For example, the display may show an outline of a hand, and then, after the patient places their hand on the display, a message may appear on the display indicating whether the hand is correctly placed or whether the hand should be moved. The message regarding body part movement may be a message telling the patient to move their body part up, down, left, or right. Alternatively, the message may be an arrow indicating the direction in which the body part should be moved to achieve correct body part placement.

[0030] In one embodiment, the display is a touch sensitive display.

[0031] As a result, a more space-efficient solution is achieved because no additional space is required for a mouse or keyboard. The display can then function as both an input and output device. For example, information about the patient to be scanned may be provided directly to the touch-sensitive display. For example, the patient can input their gender and age into the touch-sensitive display, which may then be transmitted to the controller. The controller can use the received input in generating movement instructions for the robotic arm and / or to search for patterns and / or contours displayed by the display. In one embodiment, the display is provided with an oleophobic coating or other optical coating that reduces the visible effect of fingerprint oil. The patient can interact with the touch-sensitive display to select when to start the scan or to indicate readiness for the scan, i.e., that their hands are positioned and the patient feels ready to scan. The operator can interact with the touch-sensitive display to view and / or manipulate acquired ultrasound images. A scan or other procedure performed by the robotic system can be started and / or stopped by providing input to the touch-sensitive display; for example, a portion of the display can function as a start and / or stop button.

[0032] In one embodiment, the system further comprises a controller for controlling the movement of the robotic arm.

[0033] The controller comprises any circuitry and / or device suitably adapted to perform the functions described herein. The controller may comprise a general-purpose or special-purpose programmable microprocessor such as a digital signal processor (DSP), application specific integrated circuit (ASIC), programmable logic array (PLA), field programmable gate array (FPGA), special-purpose electronic circuitry, etc., or a combination thereof. The controller may also be provided with a receiver, transmitter, and / or transceiver for receiving and transmitting signals via either a wired or wireless connection.

[0034] The controller may be communicatively connected to the screen to send signals to and / or receive signals from the display. The controller may be communicatively connected to one or more sensors of the robotic system to send signals to and / or receive signals from the one or more sensors. The controller may receive input from an operator to start a scan, stop a scan performed by the robotic arm, or otherwise modify a scan performed by the robotic arm. The controller may be provided with input means that allow an operator to provide input to the controller. The input means may be a touchscreen, a keyboard, and / or a mouse. The input means may be configured to receive input via voice or action, such as voice control, eye movement, or gesture. The controller may comprise data storage. The data storage may store one or more movement instructions for the robotic arm corresponding to one or more different body parts. The data storage may store contours of the body parts. The data storage may store instructions for a calibration process. The data storage may store a patient profile. The controller may be configured to generate movement instructions for moving the robotic arm based on data received from the sensors and / or the display.

[0035] In one embodiment, the robotic system comprises: The display is configured to detect, via the display, a contour of at least a portion of a body part supported by the display.

[0036] As a result, the robotic system obtains information about the body part that can be used in moving the robot or to identify and / or classify the body part.

[0037] The contour of at least a portion of the body part may be detected by the display, which may be a touch-sensitive display capable of detecting the contour of an object touching the display. The contour of at least a portion of the body part may be detected by the display in cooperation with a sensor. The sensor may be a camera configured to acquire image data of the support surface. The display may be configured to run a pattern and / or a color sequence. A controller may be configured to receive the acquired image data and process the image data to ascertain which pixels deviate from the pattern and / or color sequence displayed by the display. Based on the deviating pixels, the controller can determine the contour of at least a portion of the body part supported by the display.

[0038] A contour of at least a portion of the detected body part may be transmitted or calculated by a controller to control movement of the robotic arm. The detected contour may be used to control the robotic arm, for example, the controller may generate or adapt movement commands based on the detected contour of the body part.

[0039] The contour of at least a portion of the body part may be stored in a patient profile associated with the patient.

[0040] In one embodiment, the robotic system comprises: The display is configured to detect, via the display, the movement and / or presence of at least a portion of a body part supported by the display.

[0041] As a result, the robotic system can adapt to changes in the position of the body part. Furthermore, the presence of the body part gives a clear indication of whether scanning is ready to begin, further facilitating an autonomous robotic system.

[0042] The presence of a body part should be understood as whether or not at least a part of the body part is supported by the display.

[0043] The movement and / or presence of at least a portion of the body part supported by the display may be detected by the display being a touch-sensitive display capable of detecting the movement and / or presence of an object contacting the display. The movement and / or presence of at least a portion of the body part may be detected by the display in cooperation with a sensor. The sensor may be a camera configured to acquire image data of the support surface, and the display may be configured to run a pattern and / or color sequence. The controller may be configured to receive the acquired image data and process the image data continuously or at time intervals to determine which pixels deviate from the pattern and / or color sequence. Based on the deviating pixels over time, the controller can determine the movement of at least a portion of the body part supported by the display. Alternatively, the movement of at least a portion of the body part supported by the display may be detected solely through the use of a sensor. The sensor may be a camera configured to acquire image data of the support surface and at least a portion of the body part supported by the display, and the acquired image data may then be transmitted to the controller for analysis, and the controller can determine whether at least a portion of the body part supported by the display has moved based on the analysis of the received image data.

[0044] The detected presence and / or movement of at least a portion of the body part may be transmitted or calculated by a controller to control movement of the robotic arm. The detected presence and / or movement may be used in controlling the robotic arm, for example, the controller may adapt movement commands of the robotic arm in real time according to the detected presence and / or movement.

[0045] The presence and / or movement of at least a portion of the body part can be used as an emergency stop of the scan. In one embodiment, the robotic system is configured to stop the automated scan if the presence of at least a portion of the body part is not detected or if excessive movement of at least a portion of the body part is detected. Excessive movement may be defined as at least a portion of the body part moving more than 1 cm, 2 cm, 3 cm, 4 cm, or 5 cm.

[0046] In one embodiment, the system further comprises a 2D sensor and / or a 3D sensor configured to acquire data related to a body part of the patient supported by the support surface and / or data related to a robotic arm having an ultrasound probe.

[0047] As a result, additional data is collected that can assist in controlling the robotic arm and / or classifying the body part.

[0048] The 2D sensor and / or the 3D sensor may be a camera and / or a LIDAR sensor. In some embodiments, the robotic system comprises a sensor assembly comprising a plurality of 2D sensors and / or a plurality of 3D sensors. In some embodiments, the 3D sensor is comprised of at least two 2D sensors. For example, the 3D sensor can be comprised of two 2D cameras configured to acquire image data of the support surface and the body part supported by the support surface at different angles. The image data acquired by the two cameras can be transmitted and processed by the controller to acquire 3D data regarding the body part supported by the support surface.

[0049] The 2D and / or 3D sensors may be configured to acquire depth data of the body part supported by the support surface, and the depth data may be used by the robotic system to create a depth map of the body part supported by the support surface. The depth map may be calculated or transmitted to a controller configured to control movement of the robotic arm. The controller may use the received or calculated depth map in controlling the movement of the robotic arm.

[0050] The 2D and / or 3D sensors may be configured to acquire data regarding the position and / or orientation of the ultrasound probe. The 2D and / or 3D sensors may be configured to acquire data regarding the position and / or orientation of the ultrasound probe relative to a body part supported by a support surface. The position and / or orientation of the ultrasound probe may be transmitted to a controller configured to control movement of the robotic arm. The controller may use the received position and / or orientation of the ultrasound probe in controlling movement of the robotic arm.

[0051] The data acquired by the 2D and / or 3D sensors may be processed by the controller to classify the body part being scanned, for example, whether a left or right hand is being or will be scanned.

[0052] The 2D and / or 3D sensors may include receivers, transmitters, or transceivers for sending and receiving signals. The 2D and / or 3D sensors may be communicatively connectable to a controller of the robotic system. The controller may be configured to generate one or more control signals for controlling movement of the robotic arm based on signals received from the 2D and / or 3D sensors. For example, the 2D and / or 3D sensors may acquire depth data of the body part to be scanned to create a depth map of the body part to be scanned. The depth data may be acquired before scanning begins or during scanning. The depth data for creating the depth map may be transmitted to the controller, which creates the depth map of the body part to be scanned in response to receiving the data. The controller can then generate or adjust movement instructions for scanning the body part according to the depth map. Alternatively, or in combination, the controller may receive one or more signals from the 2D and / or 3D sensors during or before scanning the body part. The signals may indicate the position and / or orientation of the ultrasound probe relative to the body. The controller can then adjust the movement commands being executed in response to signals that indicate the position and / or orientation of the ultrasound probe relative to the body part being or to be scanned.

[0053] In one embodiment, the robotic system is configured to utilize a display in the calibration process.

[0054] As a result, controlled calibration of the robotic system may be achieved, allowing the robotic system to operate with high precision.

[0055] In one embodiment, the robotic system is configured to utilize calibration patterns and / or calibration colors displayed by the display in the calibration process.

[0056] The calibration pattern and / or calibration color may be a still image displayed by the display. The calibration pattern and / or calibration color may be a series of images and / or patterns displayed by the display. The calibration pattern and / or calibration color may be used to display a single calibration point for use in calibrating the robotic system. The calibration pattern and / or calibration color may be used to display multiple calibration points for use in calibrating the robotic system.

[0057] The calibration process may be initiated by an operator providing a start input to the controller. The calibration process following the start input may be controlled by the controller or by the controller in conjunction with the operator.

[0058] In one embodiment, the robotic system is configured to move the robotic arm to contact a calibration pattern and / or a calibration color displayed by the display during the calibration process.

[0059] Calibration of the robotic arm may be performed by an operator moving the robotic arm to contact one or more calibration points defined by a calibration pattern and / or calibration color. The operator can specifically move an ultrasound probe held by an end piece of the robotic arm to contact one or more calibration points defined by a calibration pattern and / or calibration color. Once the robotic arm contacts one or more calibration points, the operator can input to the robotic system that one or more calibration points have been reached.

[0060] Calibration of the robotic arm may be performed by a controller for controlling movement of the robotic arm in cooperation with 2D and / or 3D sensors configured to acquire data regarding a patient's body part supported by a support surface and / or data regarding the robotic arm having an ultrasound probe. The 2D and / or 3D sensors may transmit information to the controller regarding where the robotic arm is located relative to one or more calibration points defined by the calibration pattern and / or calibration color, e.g., where an ultrasound probe held by an end piece of the robotic arm is located relative to one or more calibration points defined by the calibration pattern and / or calibration color. The controller may generate movement instructions for the robotic arm to move toward one or more calibration points defined by the calibration pattern and / or calibration color in response to the information received from the 2D and / or 3D sensors. The 2D and / or 3D sensors may transmit information to the controller when the robotic arm reaches one or more calibration points defined by the calibration pattern and / or calibration color, e.g., whether an ultrasound probe held by an end piece of the robotic arm has reached one or more calibration points defined by the calibration pattern and / or calibration color.

[0061] In one embodiment, the robotic system is configured to calibrate the 2D and / or 3D sensors using calibration patterns and / or calibration colors displayed by the display. Calibrating the 2D and / or 3D sensors allows for mapping of 3D coordinates in space to 2D image coordinates. Calibration may be performed by displaying a calibration pattern with known parameters of size, shape, and / or position via the display. The 2D and / or 3D sensors can then acquire image data of the displayed calibration pattern and / or calibration colors. The image data may be correlated with the known parameters of the calibration pattern and / or colors. This may be a fully automatic process in which a controller communicatively connected to the display and the 2D and / or 3D sensors is configured to receive and correlate image data from the 2D and / or 3D sensors, including the known parameters of the calibration pattern and / or colors. The correlation by the controller may be to obtain a transformation matrix that allows for mapping of 3D coordinates in space to 2D image coordinates. Thus, image data acquired by 2D and / or 3D sensors may be used to accurately quantify different geometric metrics in 3D space.

[0062] In one embodiment, the display surface is configured to support at least a portion of the body part, the display surface extending within the display surface at an angle relative to the horizontal plane of between 0 and 75 degrees, preferably between 0 and 60 degrees, and even more preferably between 0 and 45 degrees.

[0063] Tilting the display surface relative to the vertical increases the display's ability to support at least a portion of a body part.

[0064] A display surface is to be understood as the surface of a display capable of displaying patterns and / or colors.

[0065] In one embodiment, the robotic system is a mobile unit.

[0066] As a result, a convenient transportation solution is provided that can be moved around rather than being fixed to a single location.

[0067] The robotic system may be provided as a trolley or a table on wheels.

[0068] It should be noted that the present invention relates to all possible combinations of features set forth in the claims. Other objects, features, and advantages of the inventive concept will become apparent from the following detailed disclosure, the appended claims, and the drawings. A feature described in connection with one of the aspects may be incorporated in other aspects, and the advantages of the feature are applicable to all aspects in which it is incorporated.

[0069] In the following description, embodiments of the present invention will be described with reference to schematic diagrams. [Brief explanation of the drawings]

[0070] [Figure 1] FIG. 1 is a block diagram of a robotic system according to an embodiment of the present invention. [Figure 2] 1 is a schematic perspective view of a robot system according to an embodiment of the present invention. [Figure 3] FIG. 3 is a schematic side view of the robot system of FIG. 2. [Figure 4] FIG. 1 is a flow diagram of a calibration process for a robotic system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0071] The present invention now will be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather, these embodiments are provided for thoroughness and completeness.

[0072] Referring initially to FIG. 1 , a block diagram of a robotic system 1 according to one embodiment of the present invention is depicted. The robotic system 1 is configured to perform automated ultrasound scanning of a patient's body part. The robotic system 1 includes a support surface 2 for supporting the body part to be scanned. The support surface 2 is adapted to receive the body part and support the body part during scanning performed by the robotic system 1. At least a portion of the support surface 2 is configured by a display 3. The display 3 is adapted to support at least a portion of the body part to be scanned. Thus, if a patient needs to perform a hand scan with the robotic system 1, the support surface 2 can support the entire hand, and the display 3, which constitutes at least a portion of the support surface 2, can support the fingers or palm of the hand. The display 3 can completely constitute the support surface 2, in which case the display 3 is adapted to support the entire body part to be scanned.

[0073] To perform the automated scanning, the robotic system 1 is provided with a robotic arm 4. The robotic arm 4 is configured to hold an ultrasound probe 8 and move the ultrasound probe 8 to obtain an automated ultrasound scan of a body part supported by the support surface 2.

[0074] In the illustrated embodiment, the display 6 is provided with a display processing device 6. The display processing device 6 is communicatively connected to the controller 5. The controller 5 is for controlling the movement of the robotic arm 4. Furthermore, the controller 5 is communicatively connected to a robotic arm processing device 7 on which the robotic arm 4 is provided. Furthermore, the controller 5 is communicatively connected to a 3D sensor 9. The 3D sensor 9 is configured to acquire data related to the support surface 2, the body part supported by the support surface 2, and / or the robotic arm 4. The controller 5 can receive signals from the display processing device 6, the robotic arm processing device 7, and / or the 3D sensor 9. The controller 5 can transmit signals to the display processing device 6, the robotic arm processing device 7, and / or the 3D sensor 9. Preferably, the controller 5 receives data from the 3D sensor 9 and / or the display processing device 6 and generates movement commands for controlling the movement of the robotic arm 4. After the movement commands are generated, they are transmitted to the robotic arm processing device 7. After receiving the generated movement commands from the controller 5, the robotic arm processing device 7 can execute the movement commands to move the robotic arm 4 in accordance with the received movement commands.

[0075] Referring to FIG. 2, a schematic perspective view of a robotic system 10 according to one embodiment of the present invention is depicted. In the illustrated embodiment, the robotic arm 14 is provided as an articulated arm 14. The articulated arm comprises a plurality of joints 141, 142, 143, 144, 145, and 146. The articulated arm 14 is connected to the system housing 101 via an end joint 141 that is connected to a base 148 of the robotic arm. The other end joint 146 of the articulated arm 14 is provided with a holder 147. The holder 147 is for holding an ultrasound probe. During scanning, the articulated arm 14 moves the holder 147 to obtain a scan of a body part. The joints 141, 142, 143, 144, 145, and 146 of the articulated arm 14 may be rotatable independently of one another, thus enabling the articulated arm 14 to perform a wide variety of movements.

[0076] In the illustrated embodiment, the body part being scanned is supported by a display 13. The display 13 constitutes a support surface in the illustrated embodiment. The display 13 comprises a display housing 131 and a display surface 132. The display surface 132 is for displaying patterns and / or colors. The display housing 131 houses the display surface 132. The display surface 132 is provided as a substantially flat surface. The display housing 131 is, in turn, connected to the system housing 101. The display housing 132 may house a display processing unit 6 and / or other electronic devices. The display 13 is a touch-sensitive display 13, i.e., an operator or a patient can provide direct input to the robotic system 10 via the display surface 132. The input provided to the display surface 132 may be transmitted to a controller 5 of the robotic system 10. The controller 5 is for controlling the movement of the articulated arm 14. The display 13 in the illustrated embodiment is configured to display an instruction pattern 133. The instruction pattern 133, in the illustrated embodiment, is an outline of the body part 133 to be supported by the display 13. The outline of the body part 133 instructs the patient where to place the body part on the display 13. The outline of the body part 133 to be displayed is patient-dependent, i.e., if the patient is an adult female and her right hand is to be scanned, the patient or the operator of the robotic system 10 can provide input to the display 13 or controller 5 that the patient is an adult female and her right hand is to be scanned, and then the display 13 can display the corresponding outline of the body part 133.

[0077] The 3D sensor 19 is connected to the display housing 131. The 3D sensor 19 includes a sensor stand 192 and a 3D sensor unit 191. The sensor stand 192 is connected to the display housing 131 and extends vertically upward from the display housing 131. The 3D sensor unit 191 is disposed on the sensor stand 192. The 3D sensor unit 191 is positioned vertically above the display 13 within the sensor stand 192. The 3D sensor unit 191 is configured to acquire data related to the display 13, a body part supported by the display 13, and / or the articulated arm 14. The data acquired by the 3D sensor unit 191 may be data related to the position and / or orientation of the articulated arm 14 or the body part supported by the display 13. The data acquired by the 3D sensor unit 191 may be 3D data of the body part supported by the display 13 and / or 3D data of the articulated arm 14. The 3D sensor unit can transmit the acquired data to the controller 5 of the robot system 10. The controller 5 can use the received data to control the movement of the articulated arm 14 .

[0078] The robotic system 10 in the illustrated embodiment is a mobile unit 10. The mobile unit 10 is achieved by connecting a robotic arm 14 and a display 13 to a system housing 101. The system housing 101 can house a controller 5 and / or other electronic devices usable by the robotic system 10. The system housing 101 is mounted on a number of wheels 102. The wheels 102 allow the robotic system 10 to be moved.

[0079] The robotic system 10 may include a controller 5 for controlling the movement of the articulated arm 14. The controller 10 may be contained within a system housing 101 or may be located remotely from the rest of the robotic system 10. The controller 5 may receive input from the articulated arm 14, the display 13, and / or the 3D sensor 19. The controller may send commands to the articulated arm 14, the display 13, and / or the 3D sensor 19.

[0080] The robotic system 10 is configured to detect, via the display 13, the contour of the body part supported by the display 13. The contour of the body part may be detected by the display 13, which is a touch-sensitive display 13 capable of detecting the contour of an object that touches the display 13. The contour of the body part may be detected by the display 13 in cooperation with a 3D sensor 19. The 3D sensor 19 may acquire image data of the display 13, which may be configured to run a particular pattern and / or color sequence. The acquired image data may then be transmitted to the controller 5, which is configured to process the image data to determine which pixels deviate from the particular pattern and / or color sequence and determine the contour of the body part supported by the display 13. The detected contour of the body part may be used by the controller 5 to control the movement of the articulated arm 14.

[0081] The robotic system 10 is configured to detect the movement and / or presence of a body part supported by the display 13 via the display 13. The movement and / or presence of a body part supported by the display 13 may be detected by the display 13 being a touch-sensitive display capable of detecting the movement and / or presence of an object contacting the display. The movement and / or presence of a body part may be detected by the display 13 in cooperation with a 3D sensor 19. The 3D sensor 19 may be configured to acquire image data of the display 13 and the body part supported by the display 13. The display 13 may be configured to run a pattern and / or color sequence. The image data acquired by the 3D sensor of the display 13 and the body part supported by the display 13 may be transmitted to the controller 5. The controller 5 may be configured to process the image data to determine which pixels deviate from the pattern and / or color sequence over time. Based on the deviating pixels over time, the movement of the body part supported by the display may be determined by the controller 5.

[0082] The detected presence and / or movement of the body part may be used in controlling the robotic arm, for example, the controller 5 may adapt the movement of the articulated arm 14 in real time according to the detected presence and / or movement. Furthermore, the presence and / or movement of the body part may be used by the controller 5 as an emergency stop. The robotic system may be configured to stop scanning if excessive movement of the body part is detected. Excessive movement may be defined as the body part moving more than 1 cm, 2 cm, 3 cm, 4 cm, or 5 cm.

[0083] Referring to FIG. 3, a schematic side view of the robot system 10 of FIG. 2 is depicted. The display 13 extends at an angle A1 with respect to a horizontal plane H. Preferably, the angle A1 is between 0 and 75 degrees, more preferably between 0 and 60 degrees, and even more preferably between 0 and 45 degrees. One end of the display housing 131 is provided with a display connection structure 132 that facilitates connection between the display 13 and the system housing 101. The display connection structure 132 is provided in an L-shape within the display housing 131, and the L-shape is configured to connect to the system housing 101. Furthermore, the articulated arm 14 includes a base 148. The base 148 is connected to a connection surface 103 of the system housing 101. The connection surface 103 is inclined downward with respect to the horizontal plane H, and the connection angle A2 is between 0 and 60 degrees, more preferably between 0 and 45 degrees, and even more preferably between 0 and 30 degrees. The connection angle A2 allows the articulated arm 14 to be lowered, thereby reducing the effective height of the robot system and resulting in a more compact robot system 10.

[0084] Referring to FIG. 4, a flow diagram of a calibration process 20 for the robotic system 1, 10 according to one embodiment of the present invention is depicted. A first step 21 involves initiating the calibration process 20. Initiation of the calibration process 20 may be performed by a controller 5 configured to calibrate the robotic system 10 before a scan of a body part is initiated. Initiation of the calibration process 20 may also be performed periodically by the controller when a scan of a body part is not being performed. Initiation of the calibration process 20 may also be performed in response to an operator input to the controller 5 that a calibration process should be initiated.

[0085] A second step 22 includes displaying the calibration pattern and / or calibration color by the display 3, 13. The display 3, 13 may be configured to display the calibration pattern and / or calibration color in response to receiving a signal from the controller 5 that the calibration process 20 has begun. The calibration pattern and / or calibration color may be a still image displayed by the display 3, 13. The calibration pattern and / or calibration color may be a series of images and / or patterns displayed by the display 3, 13. The calibration pattern and / or calibration color may be used to display one calibration point for use in calibrating the robotic system. The calibration pattern and / or calibration color may be used to display multiple calibration points for use in calibrating the robotic system.

[0086] The calibration process 20 may be performed to calibrate different components of the robotic system 1, 10. If the robot arm 4, 14 of the robotic system 1, 10 is to be calibrated, the third step 23 and the fourth step 24 may be performed. If the 3D sensor 9, 19 of the robotic system 1, 10 is to be calibrated, the fifth step 25 and the sixth step 26 may be performed instead of the third step 23 and the fourth step 24. Alternatively, both the robot arm 4, 14 and the 3D sensor 9, 19 may be calibrated in parallel, and thus the third step 23 and the fourth step 24 are performed in parallel with the fifth step 25 and the sixth step 26.

[0087] A third step 23 includes moving the robot arm 4, 14 to contact the calibration pattern and / or calibration color displayed by the display 3, 13. Calibration of the robot arm 4, 14 may be performed by an operator moving the robot arm 4, 14 to contact one or more calibration points defined by the calibration pattern and / or calibration color. The operator may specifically move the ultrasound probe 8 held by the end piece 146 of the robot arm 4, 14 to contact one or more calibration points defined by the calibration pattern and / or calibration color. Alternatively, the movement of the robot arm 4, 14 to contact one or more calibration points defined by the calibration pattern and / or calibration color may be determined by the controller 5. The controller may send movement commands including the determined movements to the robot arm 4, 14 to move the robot arm 4, 14 according to the determined movements. The controller 5 can determine the movement in cooperation with a 3D sensor 9, 19 configured to acquire image data of the support surface 2 and image data on the robot arm 4, 14 using the ultrasound probe 8. The 3D sensor 9, 19 can transmit the image data to the controller 5. The controller 5 is configured to determine, based on the image data received from the 3D sensor 9, 19, where the robot arm 4, 14 is located relative to one or more calibration points defined by the calibration pattern and / or calibration color. The controller can then generate movement instructions based on the position of the robot arm 4, 14 relative to the one or more calibration points defined by the calibration pattern and / or calibration color. The movement instructions include movements required for the robot arm 4, 14 to reach the one or more calibration points defined by the calibration pattern and / or calibration color. The movement instructions are transmitted to the robot arm 4, 14, and the robot arm 4, 14 then executes the movements. The movement instructions generated by the controller 5 can be movement instructions for moving the end piece 146 of the robot arm 14 toward one or more calibration points. The movement commands generated by the controller 5 may be movement commands to move the holder 147 of the robot arm 14 towards one or more calibration points.The movement commands generated by the controller 5 may be movement commands to move the ultrasound probe 8 held by the robotic arm 14 towards one or more calibration points.

[0088] A fourth step 24 includes verifying that the robot arm 4, 14 has reached one or more calibration points defined by the calibration pattern and / or calibration color. Verification may be performed by an operator inputting to the robot system that the robot arm 4, 14 has reached one or more calibration points. Alternatively, the controller 5 can verify whether the robot arm has reached one or more calibration points by analyzing image data received by the 3D sensors 9, 19.

[0089] A fifth step 25 comprises acquiring image data by the 3D sensors 9 , 19 and transmitting the acquired image data to the controller 5 .

[0090] A sixth step 26 involves correlating, by the controller 5, the received image data with one or more known parameters of the calibration pattern and / or color displayed by the display 3, 13. The one or more known parameters may be stored in a data storage communicatively connected to the controller 5. The correlation by the controller 5 may be to obtain a transformation matrix that allows for the mapping of 3D coordinates in space to 2D image coordinates.

[0091] Furthermore, variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. 1. A robotic system for performing an ultrasound scan on a body part of a patient, comprising: a support surface for supporting the body part; The display and a positioning device configured to hold an ultrasound probe and move the ultrasound probe to obtain the ultrasound scan of the body part supported by the support surface; wherein the display is integrated into the support surface and adapted to support at least a portion of the body part to be scanned.

2. The robotic system of claim 1 , wherein the display is configured to display an instruction pattern.

3. The robot system of claim 1 or 2, wherein the display is a touch-sensitive display.

4. The robot system of claim 1 , further comprising a controller for controlling movement of the positioning device.

5. detecting, via the display, a contour of the at least a portion of the body part supported by the display; The robot system according to claim 1 , wherein the robot system is configured as follows:

6. and detecting, via the display, the motion and / or presence of the at least a portion of the body part supported by the display. The robot system according to claim 1 , configured as follows:

7. 7. The robotic system of claim 1, further comprising a 2D sensor and / or a 3D sensor configured to acquire data related to the body part of the patient supported by the support surface and / or data related to a robotic arm having the ultrasound probe.

8. The robotic system of claim 1 , configured to utilize the display in a calibration process.

9. 9. The robotic system of claim 1, wherein a viewing surface of the display is configured to support at least a portion of the body part, the viewing surface extending within the viewing surface at an angle relative to a horizontal plane of 0 to 75 degrees, preferably 0 to 60 degrees, and even more preferably 0 to 45 degrees.

10. 10. The robot system of claim 1, which is a mobile unit.

Images